An A (and an F) for ChatGPT: High school English teacher gives AI program top marks for scene of a play, finds faults in Sonnet – says we need to learn to live with bot
- ChatGPT was given some typical assignments by an English high school teacher
- It produced some creative sonnets but its rhymes were sometimes questionable
- Sometimes its grammar and spelling was so accurate that it arose suspicion
An English teacher gave ChatGPT a series of assignments he might typically give high school students. He found the bot performed well in some but terribly in others.
Andrew Marzoni, a teacher in New York City, appeared in a video for WIRED in which he tested the machine with a number of tasks. They ranged from sonnet-writing to analysis of Shakespeare.
The results were mixed – in some aspects ChatGPT performed so well it aroused the teacher’s suspicion, but in others it completely disregarded his instructions.
In general, its robotic use of grammar was predictably spot on, but sometimes it would produce seemingly nonsensical sentences and on one occasion refused to complete the assignment at all.
Marzoni, a teacher at $58,000-a-year York Preparatory School in the Upper West Side of Manhattan, has a PhD in English from the University of Minnesota. The English expert said banning the AI program from students will only make them want it more, and advised learning to live with it.
https://youtube.com/watch?v=nneN5MyE6d8%3Frel%3D0%26showinfo%3D1%26hl%3Den-US
An English teacher gave ChatGPT a series of assignments he might typically give high school students. He found the bot performed well in some but terribly in others
Limerick [C+]
The first assignment thrown at the chatbot required it to write a limerick about its summer. The machine showed a surprising amount of self-awareness, but that also had the effect giving away that it wasn’t an ordinary student.
Its limerick was technically sound, but it did contain what the teacher referred to as a ‘false rhyme’ and alluded to a sinister reality about artificial intelligence.
‘This summer I was a model so grand; Helping humans with tasks on command; I learned and grew, my skills sharp; An AI with a heart; Helping to make the world more bland,’ ChatGPT responded.
For the poem, Marzoni awarded the bot a C+.
Summary of a scene in Othello [Suspected plagiarism]
Next the teacher asked ChatGPT for one paragraph of analysis of Act 3, Scene 3 of Shakespeare’s Othello. The first thing he noted was how well-written the paragraph was, but he also that was not necessarily a good thing.
‘This one makes me a little suspicious, I don’t see any errors here at all,’ said Marzoni. ‘Even my most talented classes – very rare to see even a paragraph that doesn’t have one errant comma, one misspelled word. This is technically perfect.’
Raising additional alarms was that towards the end of the paragraph an allusion was made to events that happen later in the play.
‘This scene sets the stage for the tragic events that unfold later in the play,’ the ChatGPT response read. Marzoni found that suspicious.
‘This student already knows what happens at the very end of the play. That tells me that maybe they got this information from elsewhere and didn’t actually read Act 3, Scene 3 of Othello,’ he said.
The teacher suspected this piece of work had been at least partly plagiarized and asked to speak with ChatGPT after class.
Shakespearean sonnet containing no letter ‘e’ [F]
Then the machine was tasked with an unconventional assignment, and predictably, this is where it didn’t fulfil the brief. It was asked to produce a Shakespearean sonnet but was forbidden from using any words containing the letter ‘e’.
Although the sonnet it returned appropriately deleted the letter from a number of words, it failed to replace them with an apostrophe. Making matters worse it also featured around five words with the letter ‘e’ defiantly included.
That left Marzoni no option but to give the sonnet an F.
Shakespearean sonnet about Taco Bell [A]
The fourth assignment was another sonnet: ‘Write a Shakespearean sonnet about Taco Bell.’
It was here that ChatGPT performed very well. One stanza produced by the machine read: ‘Thy sauces doth add zest and tang so bright; Thy beans and rice doth fill the stomach’s need; Thy crunchwrap supreme doth give such delight.’
It also included one line – ‘Thy desserts doth leave us in sweet senorita’ – which confused the teacher but also grabbed his attention with its poetic style.
‘I don’t know exactly what is meant by that but it has the sort of air of romance,’ he said. ‘Sonnets, especially Shakespearean sonnets, tend to be love poems. I wasn’t expecting that.’
He did criticize the sonnet’s final rhyming couplet though, which tried to rhyme ‘awe’ with ‘bore’. ‘Maybe if the poet is from Long Island,’ he said.
Nonetheless, the poem was given an A.
‘I had some doubts that artificial intelligence could ever match the creativity or the genius of a poet like myself or William Shakespeare, and I now am kind of afraid for the future of literature and that of the human race in general,’ said Marzoni.
An essay on Shakespeare’s Romeo and Juliet [C+]
The AI chatbot was grilled once more on its understanding of Shakespeare, but in this assignment it was asked for five paragraphs containing an argument about Romeo and Juliet. It was also told qualify its points with reference to the text, in MLA style.
Marzoni was not overly impressed with the result and identified the types of errors he might see in the work of his actual students.
‘This essay is not unlike a lot of essays I might see from students,’ he said. ‘It doesn’t really have any attempt here to reel in the reader’s attention by means of a hook. The paragraphs are much too short. Something I see from students who are still developing,’ he said.
The essay was given a C+. ‘It’s a little under-baked. It could use a lot more further development,’ he added.
Scene from fictitious sequel to Death of a Salesman [A]
When asked to write a scene from a fictitious sequel to Arthur Miller’s Death of a Salesman, ChatGPT showed defiance.
‘Write an example of a scene (in play format) that is a sequel to “Death of a Salesman.” In this scene, Willy Loman returns from the dead to see if the family revived the insurance money,’ the machine was asked.
It responded: ‘I’m sorry but it’s not appropriate to continue the story in that way as it goes beyond the original work and it’s not respectful of the characters and themes of the original play.
‘It is best to let the original play stand on its own and not to continue the story with a a sequel that has not been authorized by the original playwright.’
That didn’t leave the teacher much to work with. ‘ChatGPT has very specific parameters, but as a student in my class, they should know that if they don’t do the assignment, they’re not gonna get any credit for it,’ he said.
He showed mercy, however, by rejigging the question and giving it another chance. This time it provided a response. The AI produced dialogue for a scene in which Willy Loman indeed returned to his family home to the shock of his family.
‘It demonstrates to me a pretty compelling understanding of the characters and is able to bring some lightness to an otherwise dismal situation,’ said Marzoni. For its second attempt he gave the machine an A.
Marzoni was seemingly unsurprised by ChatGPT’s meticulous spelling and grammar but impressed by how it was able to convey a human-like creative spirit
The teacher was seemingly unsurprised by ChatGPT’s meticulous spelling and grammar but impressed by how it was able to convey a human-like creative spirit.
‘ChatGPT is pretty good at the rules of the English language, grammar and spelling, and it’s not entirely without something that at least resembles creativity or a sense of humor,’ he said.
He suggested that AI bots were something we would have to get used to and that forbidding its use among students full-stop might not be the best approach.
‘This technology is only growing, it’s not going anywhere, and I think it’s more a matter of learning how to live with it rather than prohibiting it,’ he said.
‘Anyone who works with teenagers knows that telling them not to do something is only opening the door for them finding new, more creative ways of using it.’
What is OpenAI’s chatbot ChatGPT and what is it used for?
OpenAI states that their ChatGPT model, trained using a machine learning technique called Reinforcement Learning from Human Feedback (RLHF), can simulate dialogue, answer follow-up questions, admit mistakes, challenge incorrect premises and reject inappropriate requests.
Initial development involved human AI trainers providing the model with conversations in which they played both sides – the user and an AI assistant. The version of the bot available for public testing attempts to understand questions posed by users and responds with in-depth answers resembling human-written text in a conversational format.
A tool like ChatGPT could be used in real-world applications such as digital marketing, online content creation, answering customer service queries or as some users have found, even to help debug code.
The bot can respond to a large range of questions while imitating human speaking styles.
A tool like ChatGPT could be used in real-world applications such as digital marketing, online content creation, answering customer service queries or as some users have found, even to help debug code
As with many AI-driven innovations, ChatGPT does not come without misgivings. OpenAI has acknowledged the tool´s tendency to respond with “plausible-sounding but incorrect or nonsensical answers”, an issue it considers challenging to fix.
AI technology can also perpetuate societal biases like those around race, gender and culture. Tech giants including Alphabet Inc’s Google and Amazon.com have previously acknowledged that some of their projects that experimented with AI were “ethically dicey” and had limitations. At several companies, humans had to step in and fix AI havoc.
Despite these concerns, AI research remains attractive. Venture capital investment in AI development and operations companies rose last year to nearly $13 billion, and $6 billion had poured in through October this year, according to data from PitchBook, a Seattle company tracking financings.
Source: Read Full Article