Why Run a Node?
🌐 Contribute to the Network
Help index the web and make it discoverable for everyone
🔒 Own Your Search
Run your own search infrastructure without depending on Big Tech
💡 Learn & Experiment
Understand how search engines work by running one yourself
🤝 Join the Community
Be part of a decentralised movement for an open web
Quick Start
-
Install Node.js
Download and install Node.js 18 or later from nodejs.org
-
Download OWD
Clone or download the repository from GitHub
git clone https://github.com/idev-games/open-web-directory.git cd open-web-directory -
Start Your Node
Run the following command:
npm start -
Access Your Node
Open your browser to http://localhost:8471
That's it! Your node is now running, crawling the web, and contributing to the network.
System Requirements
- Operating System: Windows, macOS, or Linux
- Node.js: Version 18 or later
- Disk Space: 1 GB minimum (configurable up to terabytes)
- RAM: 512 MB minimum (1-2 GB recommended)
- Network: Always-on internet connection (optional but recommended)
Configuration
Customize your node by editing src/config.js:
Storage Contribution
Set how much disk space you want to contribute:
storage: {
limitBytes: 10 * GB, // Contribute 10 GB
}
Crawler Settings
Adjust crawling speed and behavior:
crawler: {
enabled: true,
requestsPerSecond: 2, // Crawl faster
}
Join the Network
Connect to other OWD nodes:
bootstrap: [
{ nodeId: '...', host: 'node.example.com', port: 8471 }
]
Running 24/7
For maximum network contribution, run your node continuously:
- Linux: Use systemd (see DEPLOYMENT.md)
- All platforms: Use PM2 process manager
- Docker: Run in a container for easy deployment
Using PM2
npm install -g pm2
pm2 start index.js --name owd
pm2 save
pm2 startup
Make Your Node Public
To allow others to connect and search through your node:
- Configure your router to forward port 8471 to your computer
- Set up a domain name (optional)
- Use a reverse proxy like Nginx for HTTPS (recommended)
- Share your node address with the community
Security Note: OWD is designed to be safely exposed to the internet, but always follow best practices. See the deployment guide for details.
Getting Help
- Documentation: README.md
- Quick Start: QUICKSTART.md
- Deployment: DEPLOYMENT.md
- Issues: GitHub Issues
Technical Details
What Your Node Does
- Crawls websites starting from seed URLs
- Extracts titles, descriptions, and links
- Stores indexed data locally in compact JSONL format
- Shares search results with the network
- Discovers and connects to peer nodes
- Provides HTTP API for search queries
Privacy & Bandwidth
- Your node only crawls public web pages
- No personal data is collected or shared
- Bandwidth usage is configurable (default: 1 request/second)
- Respects robots.txt and crawl delays
- Storage is limited to configured amount
Contributing
Beyond running a node, you can also:
- Report bugs and suggest features
- Contribute code improvements
- Improve documentation
- Share OWD with others
See CONTRIBUTING.md for details.