OpenAI is the creator of many notable AI tools, both for users like ChatGPT, and APIs for developers that can be used to create a generative search index for a personal website, that at least one community member has implemented.
You can create a chatbot that generates answers to questions about content on your website using OpenAI's Embeddings API and GPT models. GPT-3.5-turbo performs well for this purpose, and for a lower cost than prior generative text models offered by OpenAI.
I use Facebook's faiss vector store for storing the embeddings.
I then keep a JSON file that maps the index associated with each embedding in the faiss store with a record I create during data ingestion. That record includes a URL at minimum and may also include a title, a date, and other meta info.
I ingest my:
- blog posts
- docs from libraries I have written
- personal wiki pages
- breakfast and coffee wiki pages
- readmes for all my public github repos
- my bio
- and some technical writing I have done
- capjamesg has been operating a chatbot on his personal website powered by Embeddings and GPT-3.5-turbo since early March, 2023.
- Playground to test the bot: https://jamesg.blog/bot/
- Supabase Clippy: ChatGPT for Supabase Docs: How Supabase made a search engine for their documentation using Embeddings and GPT.
- Proposed "Schillace Laws of Semantic AI" from Microsoft: https://learn.microsoft.com/en-us/semantic-kernel/howto/schillacelaws
- An RSS feed for their blog (built by capjamesg, updated every five minutes): https://jamesg.blog/openai.xml
- large language model