Link to the chatbot:
Here’s the link to the data that I used to fine-tune the model:
https://docs.google.com/document/d/1X8EryUyJsmBEqllso07Rp_TAZy6Ss2kdtnTsLFgmQfE/edit?usp=sharing
I did not use exactly the same prompt as the one I had in the previous activity. I found that the one I had in the first chatbot assignment was inadequate. Therefore, I refined the prompt for assignment 3 while trying to stick with the same theme.
Speaking of fine-tuning, it took quite some effort to make it work in the way that I wanted it to. In the beginning, I was thinking of typing questions that users might ask and simply providing the responses that the assistant would answer. I tried the first run with only 10 examples. 10 examples were obviously insufficient to teach the model the pattern of how I respond, but I wanted to see whether the examples work. In addition to that, I made some adjustments to the temperature at this stage. With a little bit of temperature adjusted, some of the answers given by the chatbot become nonsense. The same thing did not happen in the previous chatbot exercise. To make sure the answers make sense, I did not make further changes to temperature or Top P.
After the first run, I started to create more examples to fine-tune the model. The fine-tuned model gives responses that sound more like me. However, the answers are very short. To improve this situation, I made changes to the responses of the assistant. I input more precise and detailed responses to the example. The responses in the examples are no longer one-word answers or one-sentence answers. I developed at least 30 questions with a variety of responses. I also created examples in alternative ways, asking the same questions with different wordings that will lead to the same outcomes. After adding these new data, and giving another run on the model, the chatbot responds like me and at the same time, unlike the first run, can provide more information than it was before. Instead of changing the data every time, I tended to add more data to the example. I think it helps the model to understand more about my expectations. Little changes are required to re-train the model and I repeated the same process several times to get to where it is now. It took some time to shape the model into what I expected.
At the end of this assignment, I realized that the chatbot is quite restricted in the way that it responds. I entered the testing data set we had in class and found that the chatbot would only answer one to two sentences. The model learned the examples from the testing data and sort of overwritten the model that it originally had. The same thing happens to my chatbot: before fine-tuning, the chatbot can provide details of some specific information that it can retrieve from the database; after fine-tuning, the chatbot completely adopts the way I will respond to questions, therefore, it will not provide as much information as it would unless follow-up requests are given. I tried to make the model adapt to the database more, unfortunately, it seems like a higher level of skills in formatting JSONL instructions is required. For now, the chatbot is only trained to give responses that sound like me and the responses given are simple.
I learned more about chatbots and the LLaMA model. Before this, I thought chatbots were completely out of my league. However, with the resources provided by OpenAI, having fun with chatbots is no longer available to people who have a full understanding of Python. This assignment shows me that we can use AI as a tool but never completely trust it. I think this assignment is interesting and I had a lot of fun working on it.