RE: Was this post WRITTEN BY MACHINE? New Harvard AI Can Recognize That

avatar
(Edited)

You are viewing a single comment's thread:

In terms of text generation, there was the recent situation with GPT2 where the researchers held back part of their model for fear it was too good.
Detection is always an arms race though. If someone deploys an AI to detect generated text, then it can become a target for adversarial attacks. In situations where there is immediate reward for getting generated text through and limited long term consequences for being caught then an adversarial attack on a detector AI only needs to be relevant for a short term. I've posted about this before in regard to Steem. I've also mentioned in regard to AI dApp projects ... if your detector AI is running where the public has cheap access to it, then it will easily be overwhelmed by an adversarial attack.
One potential solution is to put the detector AI behind a paid API - a penny or two per detection request might be sufficient to raise the cost of adversarial probing to the point most attackers won't bother. There are other countermeasures an API could take but they are not fool proof.
As an aside, adversarial attacks aren't anything particularly special in AI. One approach to creating generators is to have it compete against a detector (the adversary) in an approach called GAN Generative Adversarial Network.
An an amusing annecdote: some years back a colleague and I annoyed somebody at a trade fair by adversarially attacking their face detection/tracking system. We managed to fool it with two circles and a line hand drawn on piece paper. Systems are much better now, but will still have vulnerabilities.



0
0
0.000
8 comments
avatar

In the case of GPT2, or any general purpose language transformer, where the purpose is to just mimic human writing, I'm certain there are statistical solutions to find this type of text.

But if the text was created with (currently mythical) General AI, the text itself should be evaluated in terms of whether or not it is valuable. If it's valuable research, it doesn't matter if a person did the research or AI, again, assuming it was produced by a mind, not just a language transformer like these examples.

0
0
0.000
avatar

Thank you for you comment @inertia, that's really interesting attitude.

Well, I think that this "statistical" kind of AI also does its own research, which is not very far away from what we usually do - we are also some kind of neural network which usually creates its opinion basing on what others say or write. At the current level maybe it is closer to just transforming text of other people, but I think that as this concept evolves (not necessarily to the level of "mythical" General AI) it will be able to generate quality publication and present an unique opinion.

As long as you admit that particular text is machine-generated, I see nothing wrong in publishing and reading such text.

0
0
0.000
avatar

As long as you admit that particular text is machine-generated, I see nothing wrong in publishing and reading such text.

Yes. I think that's important. It's good to know the background of the author. If the author is reluctant to discuss their background, then they must be worried about their safety. For the time being, we can assume it's political pressure. But some day in the not too distant future, we will have to consider that an anonymous, unverified author might be on the wrong side of politics, is AI, or both.

0
0
0.000
avatar

But some day in the not too distant future, we will have to consider that an anonymous, unverified author might be on the wrong side of politics, is AI, or both.

That's right, unfortunately.

0
0
0.000
avatar

That's one of the reason's I'm publishing my autobiography online, daily, all over the place.

0
0
0.000
avatar

Thank you for sharing your opinion @eturnerx. I appreciate it a lot :)

One potential solution is to put the detector AI behind a paid API - a penny or two per detection request might be sufficient to raise the cost of adversarial probing to the point most attackers won't bother.

I totally agree with you, but I suppose GLTR is rather an experiment than a serious tool. As I mentioned, it only increases a chance to spot the fake text, and I believe that in about 6-12 months machine-generated texts will be so advanced that GLTR will become helpless. And unfortunately I see no space for further development and improvements of this tool.

An an amusing annecdote: some years back a colleague and I annoyed somebody at a trade fair by adversarially attacking their face detection/tracking system. We managed to fool it with two circles and a line hand drawn on piece paper. Systems are much better now, but will still have vulnerabilities.

That's really interesting. As far as I know, till the premiere of iPhone X face recognition systems were generally easy to trick with a photo. Ultimately Apple introduced their special sensor able to recognize face in 3 dimensions, thus becoming resistant to the images. However I think you can still convince it by using some precise mask.

0
0
0.000
avatar

I believe that in about 6-12 months machine-generated texts will be so advanced that GLTR will become helpless. And unfortunately I see no space for further development and improvements of this tool.

I think it's an arms race. While GLTR might not be the best detector in future, something else will be.

0
0
0.000
avatar

While GLTR might not be the best detector in future, something else will be.

I agree, but the concept of this tool will have to be reworked.

0
0
0.000