Automation
Learn how Dirstarter uses AI and third-party services for automating directory content generation
Dirstarter uses AI and third-party services for automating directory content generation.
How It Works
We use a set of tools to automate the content generation. This speeds up the process and ensures a consistent output.
Here's a step-by-step guide on how the automation works:
A tool with URL is added to the directory
Admin reviews the tool and schedules it for publishing
Jina AI scrapes the website and generates a structured JSON object
Google Gemini generates structured data and categorises the content
Screenshot of the website is taken using Screenshot API
Favicon is automatically extracted from the website
The generated content is saved to your database
Setup
In order to use the automation, you need to set up the following:
Jina AI
We use Jina AI to scrape the website and generate a structured JSON object. They provide a very generous free tier without an API key, but if you run out of credits, you can create a API key to get 1,000,000 free tokens.
Google Gemini
Create a Google Gemini account and add the following environment variable:
We chose Gemini as a default model because it's the most cost-effective option. If you need more powerful model, you can use Claude or any other LLM supported by Vercel AI SDK.
Customizing Generation
You can customize the generation by:
- Modifying the system prompt in the
/api/ai/generate-contentfunction - Adjusting the schema to include additional fields
- Changing the temperature parameter (lower for more consistent results)
Last updated on