BPKP Employee Leave Policy Q&A Chatbot

Problem

Employees needed a faster way to find information about leave policies contained in a BKN regulation document. Manually searching through the PDF was time-consuming, while sending the entire extracted document directly to a language model provided limited retrieval control and could become inefficient as the document collection grew.

Decision

The selected implementation used a retrieval-augmented generation pipeline. The BKN regulation PDF was extracted into text, divided into searchable document segments, converted into vector embeddings, and stored in a FAISS index. When a user submitted a question, the system embedded the query and used FAISS similarity search to retrieve the most relevant document segments before sending them to the Q&A model. The OpenAI implementation used text-embedding-3-small for embeddings and GPT-4o-mini for answer generation. A local alternative was also prepared through an Ollama Docker container containing Llama 3.2 1B for Q&A and Nomic Embed Text for embeddings.

Result

The chatbot enabled users to ask natural-language questions about employee leave regulations and receive answers based on the most relevant sections of the source document. FAISS reduced the amount of context sent to the language model by retrieving only closely related passages, creating a more focused and efficient question-answering workflow.

Retrospective

Future improvements could include source citations with page references, automated document re-indexing, retrieval-quality evaluation, hybrid keyword and vector search, conversation history, and role-based access control. Additional testing with real employee questions would also help identify ambiguous regulations and improve answer reliability.