Automation
Learn how Dirstarter uses AI and third-party services for automating directory content generation
Dirstarter uses AI and third-party services for automating directory content generation.
How It Works
We use a set of tools to automate the content generation. This speeds up the process and ensures a consistent output.
Here's a step-by-step guide on how the automation works:
A tool with URL is added to the directory
Admin reviews the tool and schedules it for publishing
Jina AI scrapes the website and generates a structured JSON object
AI generates structured data and categorises the content via Vercel AI Gateway
Screenshot of the website is taken using Screenshot API
Favicon is automatically extracted from the website
The generated content is saved to your database
Setup
In order to use the automation, you need to set up the following:
Jina AI
We use Jina AI to scrape the website and generate a structured JSON object. They provide a very generous free tier without an API key, but if you run out of credits, you can create a API key to get 1,000,000 free tokens.
JINA_API_KEY=your_api_keyVercel AI Gateway
Dirstarter uses the Vercel AI Gateway for AI-powered content generation. This allows you to use models from OpenAI, Anthropic, Google, and other providers through a single, unified API.
- Go to your Vercel Dashboard → Settings → AI Gateway
- Create an API key and add the following environment variables:
AI_GATEWAY_API_KEY=your_api_key
AI_CHAT_MODEL="openai/gpt-4o"
AI_COMPLETION_MODEL="openai/gpt-4o-mini"AI features are optional. If AI_GATEWAY_API_KEY is not set, AI generation will be gracefully disabled.
Changing AI Models
You can easily switch to different models by changing the environment variables. For example:
- Google Gemini:
google/gemini-2.5-pro - Anthropic Claude:
anthropic/claude-sonnet-4 - OpenAI:
openai/gpt-4o(default)
See the full list of supported models in the Vercel AI Gateway documentation.
Customizing Generation
You can customize the generation by:
- Modifying the system prompt in the
/api/ai/generate-contentfunction - Adjusting the schema to include additional fields
- Changing the temperature parameter (lower for more consistent results)
- Choosing different chat and completion models (via environment variables)
Last updated on