New research has claimed, the AI chat service ChatGOT is capable of replacing human workers who train AI programs. According to research and tests placed on ChatGPT, the AI chatbot is capable of being more accurate and consistent in performing text-annotation tasks than a human worker.
Machine learning systems tend to depend on workers for training and fine-tuning Artificial intelligence models, but with the latest research, it seems workers are likely to be replaced by the model itself. This has resulted in generating a huge impact on underpaid human workers who work manually by labeling and filtering content for AI datasets regularly.
Key Points:
- Underpaid workers on platforms like Mechanical Turk, who train AI models are likely to be replaced by AI models themselves.
- ChatGPT is likely to produce a higher level of accuracy and consistency in text-annotation tasks compared to underpaid human workers.
ChatGPT is Capable to Outperform Underpaid workers
Political science researchers from the University of Zurich witnessed in the latest paper, that ChatGPT has the capability to outperform underpaid crowd-workers, who perform tasks such as text annotation, which is labeling texts which are used for training purposes in an AI system.
The research found ChatGPT is capable of labeling these texts with a higher level of accuracy and consistency than human annotators, which they found on Mechanical Turk, a crowdsourcing platform owned by Amazon, which also trains annotators such as research assistants.
The researchers tested ChatGPT by asking it to classify 2,382 tweets into texts via their topic, relevance, stance, policy framing, and problem or solution flaming.
Researchers concluded by saying they found a higher level of accuracy from ChatGPT along intercoder agreement, which is the percentage of tweets that were tasked the same level by two different ChatGPT runs.
Through this test they also discovered by using ChatGPT, they could also save money since utilizing an AI chatbot is way cheaper than recruiting and paying a human on Mechanical Turk, who made about 5 cents with every annotation.
The study showcased the reality of how AI systems are affecting and making an impact on human jobs at such a fast pace by using large language models like ChatGPT.
In a recent paper, OpenAI researchers argued that 80% of the US workforce has the potential to have at least 10% of their work and tasks affected by GPTsr introduction.
Although human annotators are especially grim since this is already precarious for the workers’ populations. After the release of ChatGPT, most high-tech companies such as Microsoft and Google are working towards their technologies, progress, and speed in the AI sector. The truth is AI models do rely on underpaid human workers.
A wide range of workers is utilized manually to filter and label content from the Artificial intelligence model’s datasets. Human workers are required to work because AI cannot recognize the nuances of a photo, especially during its initial training period.
It was reported by Time magazine earlier this year, that OpenAI pays Kenyan workers to make AI chatbots safer for its users and they pay about $2 for an hour. Even after the AI model is deployed, they tend to still rely on human interaction to identify and also fine-tune the shortcomings of the AI models.
Krystal Kauffman, a Turker (referring to MTurk workers) for about seven years and is currently in Turkopticon (a non-profit organization that works for Turkers’ rights) stated the company’s Turker doesn’t believe that ChatGPT’s capabilities can replace their abilities.
ChatGPT keeps learning and changing. If the information runs on OpenAI’s latest model GPT-4, would it be able to showcase the same results? Would there be a difference in a year after the countless addition to data sets? What is the source that trains the AI model?
We also noticed the study run on ChatGPT capabilities which demonstrates a deficiency of peer review, said Kauffman. ChatGPT can create texts however, a human needs to read it and conclude whether it’s good enough. (ensuring it doesn’t contain any offensive or disrespectful content)
She added, “Writing or generating content isn’t just about creating words or judgment”. People like Turkers are essential for the current and foreseeable future, to perform judgment tasks. Currently, there are tons of unanswered questions and judgments to feel confident about choosing ChatGPT’s capabilities over human workers.
The researchers admitted it’s quite early to state to which extent ChatGPT can replace human workers. It was stated by the paper’s co-author, Fabrizio Gilardi, “The paper showcased ChatGPT’s ability to perform task-annotation tasks, including its accuracy and consistency.
More research and testing is mandatory to understand ChatGPT’s abilities in various tasks and whether it can replace workers”. For example, the test conducted on tweets in English and ChatGPT could perform a limited number of tasks. Although it is essential to expand these tests and analysis to more tasks, languages, and data”.
