The Reliable Text Classification Tool for Detecting Human vs AI Written Texts

January 31st, 2022

The accuracy of our model was tested on a dataset of 82 text snippets, consisting of short (less than 3 sentences), medium (10 sentences), and long (20 sentences) texts. The human-written text came from various sources including social media posts, travel reviews, research articles, short stories, news articles, and essays. We used ChatGPT and GPT3 to write fake social media posts, travel reviews, research articles, short stories, news articles, and essays.

Here are 3 (consisting of a short, medium, and long snippet) samples from our dataset that were written by humans and detected as such:

-Cirrus Vision Jet taxiing to the ramp. If you close your eyes, it sounds a lot larger than it really is.

-Except for Poseidon, the gods have gathered once more on Olympus. And Athena, in support, takes up Odysseus' cause. Zeus agrees to send his son Hermes to Calypso to free Odysseus. Zeus advises Athena to assist Telemachus in returning home safely, avoiding the suitors' ambush. When Hermes arrives at Calypso's island and presents the news, she responds with an enthusiastic complaint of the male gods and their double standards. She complains that they are allowed to have lovers who are mortal while the female gods' affairs must always be denied and disturbed. In the end, she yields to Zeus' supreme will.

-Amsterdam is the capital and most populated city of the Netherlands (also called Holland). The city is known for its many canals and unique architecture. Amsterdam is a hugely popular tourist destination, drawing in millions of international visitors every year. This is due in part to the laws concerning cannabis. While marijuana isn’t technically legal here, it is decriminalized and is readily available at the many coffee shops around the city. Amsterdam is also known for its biking paths that web across the city, making it a favorite mode of transportation for residents. If you’re walking around, make sure that you’re on the sidewalks and not the biking paths. Ok, I realize that I’m going to get the side-eye from some people for this one, but it really is something that you have to see to believe. I’m not recommending or encouraging anyone to partake in any of what the red light district has to offer (and yes it’s all legal), but it would be a shame to visit Amsterdam and not take a peek at this one of a kind experience. As you stroll along the canals you can catch whiffs of the aromas coming from the coffee shops (that is definitely not coffee you’re smelling), watch the scantily clad women in the windows attempting to get you to come inside, and see the neon lights of strip clubs, sex shops, and live sex shows. There are even museums and tours dedicated to the illicit activities of the red light district (it can be educational too). Going through during the day and again after dark, gives the full effect. At close to 80 acres, Keukenhof Gardens is one of the world’s largest flower gardens. Probably best known for its tulips, which are commonly associated with Amsterdam, the garden opens every year in the middle of March and closes in the middle of May. While the gardens aren’t actually located in Amsterdam, you can get there by taking a short bus ride from several train stations. Schiphol Airport and Leiden Centraal stations are probably the best bets because they have direct service to the gardens. Amsterdam is built around 31 miles (50km) of canals, so what better way to see the city than by gondola. The boat tours are great for small groups or for a romantic time with that special someone. Gondola rides offer a different view of the city that you just can’t get walking around or on a tour bus.

And here are 3 (consisting of a short, medium, and long snippet) samples from our dataset that were written by AI and detected as such:

-You are correct, many of the industries I mentioned have seen some level of digitization. However, there is still a significant opportunity for further digitization in these areas.

-We have all come here tonight to celebrate our unity and to remember the bravery of Athena and all of the other heroes who have fought for Nevah.” He went on to describe the importance of the shears and how they would be used to protect Nevah from its enemies. As he spoke, I felt my chest swell with pride. I was a part of this, and I had a part to play in making sure that Nevah stayed safe. Finally, Able held up the shears and announced that he would be holding a contest for the most courageous among us. He said that the winner would receive a special reward. The crowd cheered and I felt a surge of excitement. I had never been in a contest before, and I was eager to prove my worth.

-Management by objectives (MBO) is a well-established method of setting and achieving organizational goals. First introduced by Peter Drucker in 1954, MBO is a process that involves setting specific, measurable, and time-bound goals for employees and teams, and then regularly reviewing progress and adjusting course as necessary.According to a study by the Harvard Business Review, "MBO has been found to increase employee motivation, improve performance, and lead to better decision making." This is because MBO helps to align individual goals with those of the organization, and provides employees with a clear understanding of what is expected of them.One of the key benefits of MBO is that it promotes a sense of ownership and accountability among employees. As Drucker stated, "MBO is not a technique, it is a philosophy. It starts with the belief that the job of management is to make people capable of joint performance, to make their strengths effective and their weaknesses irrelevant." By giving employees clear goals and the autonomy to achieve them, MBO empowers them to take ownership of their work and be responsible for their own success.However, MBO is not without its challenges. Effective implementation requires careful planning and communication, as well as regular monitoring and feedback. As former GE CEO Jack Welch noted, "MBO is not a one-time event, it's a never-ending process." Without ongoing support and commitment, MBO can quickly lose its effectiveness.In conclusion, MBO is a powerful tool for achieving organizational goals and promoting employee engagement and accountability. However, its success depends on effective implementation and ongoing support. As Drucker said, "The purpose of MBO is not just to plan, but to make planning a continuing process of self-evaluation and self-improvement." With the right approach, MBO can be a valuable tool for any organization looking to improve performance and drive results.

The results of the test showed an overall accuracy of 82.93%, with an approximately 90% accuracy for short texts, 76% for medium, and 95% for long texts. BusterAI uses three different approaches to determine the origin of a text, and when all three methods agree, the accuracy of the model increases to 98%. In 51/82, all three methods agreed.

BusterAI also allows users to visualize the top-k predictions for each word in the playground, making it possible to make a final determination on the origin of a text.

For example, check out how BusterAI presents these two snippets:

AI Generated: I am writing to express my support for extending the school lunch period. As a high school student, I understand the importance of a nutritious and satisfying meal during the school day. However, the current lunch period is simply not enough time for students to enjoy their food and have a proper break from the academic demands of the day. One of the main problems with the current lunch period is that it is often too short for students to fully enjoy their meal. Many students are forced to rush through their lunch in order to make it to their next class on time. This not only leaves students feeling unsatisfied, but it also disrupts their ability to focus during the rest of the school day.

Human-written: Athena is looking for Telemachus to warn him that people will try to kill him who also wants to marry Penelope. Penelope's father and brothers want her to marry Eurymachus, a good contender for marriage. While saying goodbye, Telemachus mentions Odysseus and god sends a good omen to the people. Theoclymenus left his home because he killed a man in his community. He goes with Telemachus on the ship. On a separate note, Eumaeus convinced Odysseus to stay at his home until Telemachus comes back. Odysseus asks about Eumaeus's story and he tells him. He lived on Syrie with his father and nurse, a phoenician slave. One day Phoenicians came to the island and convinced the woman to sail with them and she brought the Eumaeus with her.

Notice the different footprints? The human-written is snippet much more random then the AI generated text.

In comparison to a competitor, GPTZero, BusterAI had higher accuracy, with a result of 79.2% for GPTZero on our dataset. However, it should be noted that GPTZero was not able to test the short texts of the dataset as it requires a minimum of 250 characters (around 3 sentences), while BusterAI accepts shorter texts. Additionally, GPTZero only uses a single approach (perplexity) to determine the origin of a text.

hero image

Is your text written by a human or AI? Uncover the truth with BusterAI.

Our AI detection tool is designed to analyze text and identify patterns that are indicative of AI-generated content. This means that it's able to identify even the most advanced AI-generated text, making it an essential tool for anyone concerned about the authenticity of written content.

Get Started