|
# Example Crawler Project
|
|
|
|
This is a simple example crawler project. Follow the instructions below to set up and run the crawler.
|
|
|
|
## Setup
|
|
|
|
1. Clone the repository or copy the project files to your local machine.
|
|
2. Make sure you have Python 3 installed.
|
|
|
|
## Usage
|
|
|
|
1. Open a terminal in the project directory.
|
|
2. Run `make` to set up the environment, install dependencies, and start the crawler:
|
|
|
|
```bash
|
|
make
|
|
```
|
|
|
|
This will create a virtual environment, install the package in editable mode from the parent directory, and run the main script.
|
|
|
|
## Cleanup
|
|
|
|
To remove the virtual environment, run:
|
|
|
|
```bash
|
|
make clean
|
|
```
|
|
|
|
## Notes
|
|
|
|
- The project installs the package with `-e ../../.` to include the parent package `devranta` in editable mode.
|
|
- Ensure that the parent package is correctly set up in the directory structure.
|
|
|
|
Happy crawling! |