A Comment Feedback Loop Architecture for Prompt Engineering

In the previous article, we present an experiment that shows highly relevant prompts can improve the performance of gpt-4 on quantum-related questions. Now the time has come to introduce the motivation and architecture of quantum-gpt. I am going to introduce a unique architecture and invite you to participate in an experiment. 

As you may have noticed, there are a few ways to make contributions. You can comment in the comment section of this article. You can also comment on the response of quantum-gpt in the app section. And last but not least, you can chat with other users in the chat section.

Your comments are going to be distilled and included in the prompt to quantum-gpt. However, we want to make sure that the comments we include are relevant and of good quality. Therefore, each comment is assigned a score with two components, relevance and credibility. Relevance score is evaluated by extracting keywords, applying labels, and using word2vec. And credibility score is directly related to the time of comment, the popularity of the comment, and your experience level. Experience can be gained in three ways:

1. Through prior (if you send an email and tell me that you are a quantum professor, I will give you a lot of experience score)

2. Through using the app and making comments (the more you are experienced with our product, the more credible your comments are)

3. By being cited or endorsed by other credible users in their comments, a small proportion of their credibility is passed along (another graph).

Detailed diagrams showing the interactions between entities are shown below:

In the above diagram, there is a prompt engineering algorithm block. The design for the prompt engineering algorithm is shown below (for simplicity we only show three comments, and credibility is associated with the user making the comment):

Last but not least, I would like to present a graph of user interactions and the credibility system. For simplicity we just include two users, each user writes a comment and the second user cites the first user's comment. The graph is shown below:

The update score calculator takes the result of the sentiment analysis, and the credibility of user2 to calculate an update score for user1's credibility. And then user1's credibility is updated accordingly.

After learning more about our architecture, we would like to invite you to do a few things to contribute to our effort. Here are some actions you can take to contribute: 

1. If you identify a mistake made in the response, please comment in detail

2. If you have a comment on the comments of the users, please explicitly cite the discussion/blog and user in the comment

3. We will not utilize any data from the chat unless it explicitly cites a discussion or a blog, citing in the chat is another channel to provide feedback to us

4. If you have an interesting topic that inspires discussion, please do not hesitate to write an article (the article will be displayed on your profile and we are going to give you lots of credibility after review)

5. Invite your colleague to use. The website is www.quantumgpt.science. You can also invite them to scan the QR code on your account page to connect with you.

6. Please let us know if you are an expert (you can either send an email to xiang@quantumgpt.science or use the teleporter on the account page to introduce yourself)

7. Please explicitly cite other discussions/articles/comments if relevant.