Avisi Apps and Labs have joined forces to participate in the monday.com AI Hackathon!

Monday.com released their AI assistant and, as a celebration, held a hackathon where we had three days to create an AI application. The challenge was to be among the first developers to build an AI app for the monday.com marketplace. Who doesn't like a challenge? We joined.

Ideation: Finding the Missing Piece

We looked at our beloved Tracket app, and a light bulb flickered above our heads. We discovered that people often start noting their work at the beginning of the week and then refer to these notes at the end of the week to fill in their time logs. This process could be automated! Additionally, automating this process would reduce the time spent on administrative tasks on Fridays—a significant benefit! Thus, our small but mighty mission team, consisting of Demi, Albert, and Jaap, embarked on a quest to boost the "net beer time" while making a meaningful contribution.

The Mission: Using AI for Logging Time

To bring our vision to life, our team harnessed the Python API and leveraged two powerful models: GPT 3.5-turbo, the chat model, and ada-002, the embeddings model. These models paved the way to extract meaning from notes and transform them into time entries.

The secret lay in comparing the embeddings, which provided a way to represent and cluster the tasks related to specific notes. All text within a task, as well as the work notes themselves, are referred to as "embeddings." Embeddings provide a way to represent strings, and tasks related to specific notes are clustered together as context for queries. By comparing the embeddings between the text and the task on the platform, we can determine their relevance.

To make the process appear as authentic as possible, they created a monday board with multiple tasks, assigned users, and status indicators. The provided input is converted into time entries, with date, item, and hours. You can watch the following demo from Demi to see how the whole process looks.

As per results, AI performed better when the prompts were detailed. If it didn't understand something, it would return an error. Providing additional information to the AI helped determine the appropriate task given the context. The saved entries were stored on your board. During the project, our team also discovered that the way you write the prompts is crucial. The AI doesn't respond as well to prompts that contain excessive negative text. For non-technical users, they realized that instructing the AI on what it should do, rather than what it should not do, produced better results.

Conclusions

In conclusion, as with any good research, we found ourselves with more questions than answers.

Challenges in Using Natural Language as Input for Applications

Using natural language input in applications is valuable, but challenges persist. Can the large language models handle all tasks effectively? Although it shows promise, there are still numerous obstacles. Item matching generally performed well, despite a few anomalies. However, the impact of a larger context on performance remains uncertain. Additionally, exploring the benefits of providing more information, like past contextual tasks, is worth investigating.

Difficulties in Controlling Output and Duration in Language Model

We also encountered difficulties in controlling the output. Although we instructed the language model to respond in JSON, there were times when it randomly switched back to responding in text. Additionally, despite requesting a default duration of eight hours when none was found, sometimes it still couldn't determine the duration.

Importance of Keeping Embeddings Up to Date for App Functioning

It is essential to keep the embeddings up to date at all times as the proper functioning of the system relies on them. The context is formed by the items on the board, which are subject to constant change. Therefore, the level of difficulty and associated costs of maintaining up-to-date embeddings are unclear.

In concluding this chapter, the importance of prompt injections cannot be overlooked. While focusing on innovation, we prioritize handling sensitive information carefully and ensuring its inclusion in prompts.

In the AI realm, we've only scratched the surface of its potential. The monday.com AI Hackathon has opened doors to transformative possibilities in our work. Avisi Apps and Labs push boundaries, inviting you to shape the future of work.

We value your feedback, and your input drives our intelligent solutions. Do you use notes for your timesheet? Share your thoughts and suggestions with us here!