Reply to: OpenAI ChatGPT

[d000hg]

Can this AI detect if it is talking to AI?

Someone must have tried plugging it back into itself. Like the takeaway game.

[AtW]

Can this AI detect if it is talking to AI?

[jamesbrown]

And, again, individual lecturers don't need to have the first clue about the threat or its detection because, in my experience, all major unis require students to submit an electronic copy of each essay and they are autoscreened and scored for plagiarism prior to marking (using AI and other techniques backed by databases).

[jamesbrown]

Have you tested that? I asked the same question days apart and got different answers. The material it read from may have changed but in something like this system, it is always changing.

It may be easy to detect now, but it's hardly been round any time at all as yet. What you can do is compare the writing style of a student's essay against something they wrote unaided in controlled conditions. That tech has been around a while. You can also quiz a student about their essay (like a viva) and if they can't answer basic questions about it, fail them.

If you're getting substantively (vs. superficially) different answers for the same question, that doesn't really lend itself to a high quality essay

As I said earlier in this thread, any technology that can be turned to answering questions can also be turned to detecting questions answered by the technology and that's indeed what is happening (and has been for ages). It isn't much different than SecOps, it's always responding to new threats but mitigating them isn't futile by any means. This isn't going to mark a step change in students getting away with cheating.

[d000hg]

I doubt they tried very hard. Afterall, it's deterministic, right? If you ask the same thing twice, you'll get largely the same answer, varying only as the source material varies.

Have you tested that? I asked the same question days apart and got different answers. The material it read from may have changed but in something like this system, it is always changing.

It may be easy to detect now, but it's hardly been round any time at all as yet. What you can do is compare the writing style of a student's essay against something they wrote unaided in controlled conditions. That tech has been around a while. You can also quiz a student about their essay (like a viva) and if they can't answer basic questions about it, fail them.

[vetran]

I know they're not slaves really, as it's a decent wage for the area, but where's the fun in that?

I'm all for it, the world is getting smaller if we work together it can get better.

[ladymuck]

Labour in a lower cost country possibly, not apparently Slaves from that story, as much of the world lives on less than a dollar a day $16 dollars seems like luxury.

Like India China as we trade more with them the standard of living will rise, Kenya will move away from poverty and war. That has to be a good thing.

We do need to impose minimum labour standards on the outsourcing so we don't have five year olds making iPhones etc.

I know they're not slaves really, as it's a decent wage for the area, but where's the fun in that?

[jamesbrown]

Apparently not. You can further tweak the answer by reading what it's done and tweaking the input so after a number iterations it's unrecognisable from the first draft. They asked this on the show.

Sure you can, it isn't a random number generator and AI can be turned easily enough to the problem of recognising other AI. Read the link I provided by way of example or watch this short video, it briefly addresses some of the things you're talking about.

https://www.youtube.com/watch?v=g85a...annel=Turnitin

And, again, these sort of tools are automatically applied to every student essay before you even look at them, the ChatGPT thing is just an evolution of the problem (and solution).

[northernladuk]

You would ask it to give you an essay plan or the major bullet points to include. And then write the essay prose yourself, in your own words. But I guess most students are even lazier or leave it to the last minute, and just want a fully complete answer.

I think we'd be surprised how much effort they'd put in to avoid doing it themselves.

[northernladuk]

I doubt they tried very hard. Afterall, it's deterministic, right? If you ask the same thing twice, you'll get largely the same answer, varying only as the source material varies. It should be easy enough to develop a cheating muppet evaluator using, say, OpenAI. These tools are already used by universities - I left university teaching a long time ago but, even back then, essays were being pre-screened by software for plagiarism and given a rating. A key feature of these tools is to compare across essays within a database for similarities, so anything roughly deterministic will be caught. It will be easy enough to improve that software too. It's always reactive, but it can react quickly.

https://www.turnitin.com/blog/sneak-...ion-capability

Apparently not. You can further tweak the answer by reading what it's done and tweaking the input so after a number iterations it's unrecognisable from the first draft. They asked this on the show.

[SueEllen]

You would ask it to give you an essay plan or the major bullet points to include. And then write the essay prose yourself, in your own words. But I guess most students are even lazier or leave it to the last minute, and just want a fully complete answer.

Seems less work to write your own essay....

[Fraidycat]

Could you not ask ChatGPT to write the essay in a certain style that would stop easy comparisions?

You would ask it to give you an essay plan or the major bullet points to include. And then write the essay prose yourself, in your own words. But I guess most students are even lazier or leave it to the last minute, and just want a fully complete answer.

[jamesbrown]

Could you not ask ChatGPT to write the essay in a certain style that would stop easy comparisions?

Yes, sure, cretin style, and then you'd get marked as a cretin.

Overall, students are pretty thick and get caught all the time. That has always been the way and technology has been a part of cheating (and catching cheats) forever.

[woohoo]

I doubt they tried very hard. Afterall, it's deterministic, right? If you ask the same thing twice, you'll get largely the same answer, varying only as the source material varies. It should be easy enough to develop a cheating muppet evaluator using, say, OpenAI. These tools are already used by universities - I left university teaching a long time ago but, even back then, essays were being pre-screened by software for plagiarism and given a rating. A key feature of these tools is to compare across essays within a database for similarities, so anything roughly deterministic will be caught. It will be easy enough to improve that software too. It's always reactive, but it can react quickly.

https://www.turnitin.com/blog/sneak-...ion-capability

Could you not ask ChatGPT to write the essay in a certain style that would stop easy comparisions?

[jamesbrown]

and was impossible to flag as plagarism

I doubt they tried very hard. Afterall, it's deterministic, right? If you ask the same thing twice, you'll get largely the same answer, varying only as the source material varies. It should be easy enough to develop a cheating muppet evaluator using, say, OpenAI. These tools are already used by universities - I left university teaching a long time ago but, even back then, essays were being pre-screened by software for plagiarism and given a rating. A key feature of these tools is to compare across essays within a database for similarities, so anything roughly deterministic will be caught. It will be easy enough to improve that software too. It's always reactive, but it can react quickly.

https://www.turnitin.com/blog/sneak-...ion-capability