# orderedItems[].object.content
@ben_golub Still worth pursuing as an important line of defense, but I don't think approaches a "full" solution

# orderedItems[].object.contentMap.en
@ben_golub Still worth pursuing as an important line of defense, but I don't think approaches a "full" solution

# orderedItems[].object.content
@ben_golub Alright just finished a read. Neat talk, and yes, this is the "fingerprinting" idea I alluded to initially. As Scott mentions (and @paulgp noted here too) it's robust to certain types of attacks, but relies on the algo author to implement AND facilitate detection. And as long as the underlying tech exists in relatively reproducible form (see growing # of "jailbroken" ChatGPT clones), non-fingerprinted algos will exist, and output can be mixed arbitrarily to conceal fingerprinted text

# orderedItems[].object.contentMap.en
@ben_golub Alright just finished a read. Neat talk, and yes, this is the "fingerprinting" idea I alluded to initially. As Scott mentions (and @paulgp noted here too) it's robust to certain types of attacks, but relies on the algo author to implement AND facilitate detection. And as long as the underlying tech exists in relatively reproducible form (see growing # of "jailbroken" ChatGPT clones), non-fingerprinted algos will exist, and output can be mixed arbitrarily to conceal fingerprinted text

# orderedItems[].object.content
@evanmiltenburg Alrighty, just got a chance to read/view. I think this is spot-on when it comes to potential automated plagiarism and pedagogy (and I think nicely aligns with my training on best practices for "regular" plagiarism when I was a HS English teacher, too).

That said, I think the value of detection is much broader than just in academic contexts, e.g., falsifying social media campaigns, generating swathes of legitimate looking news articles to disinform. And what then?

# orderedItems[].object.contentMap.en
@evanmiltenburg Alrighty, just got a chance to read/view. I think this is spot-on when it comes to potential automated plagiarism and pedagogy (and I think nicely aligns with my training on best practices for "regular" plagiarism when I was a HS English teacher, too).

That said, I think the value of detection is much broader than just in academic contexts, e.g., falsifying social media campaigns, generating swathes of legitimate looking news articles to disinform. And what then?

# orderedItems[].object.content
@evanmiltenburg @Iris
Ah, will definitely take a peek at these and get back!

# orderedItems[].object.contentMap.en
@evanmiltenburg @Iris
Ah, will definitely take a peek at these and get back!

# orderedItems[].object.content
@ben_golub Not yet! Will bookmark for after work today

# orderedItems[].object.contentMap.en
@ben_golub Not yet! Will bookmark for after work today

# orderedItems[].object.content
@paulgp Mmm that's a good call, though exactly how much human editing is required to avoid which methods and thresholds of detection is the key Q (juice, squeeze, etc.).

We're also talking an endlessly increasing variety of LLMs, which can be arbitrarily mixed together, and a growing market for LLMs without fingerprinting... So even if a major player (e.g., OpenAI) uses robust fingerprints, it may not be sufficient against many automated avoidance techs one could throw together quickly

# orderedItems[].object.contentMap.en
@paulgp Mmm that's a good call, though exactly how much human editing is required to avoid which methods and thresholds of detection is the key Q (juice, squeeze, etc.).

We're also talking an endlessly increasing variety of LLMs, which can be arbitrarily mixed together, and a growing market for LLMs without fingerprinting... So even if a major player (e.g., OpenAI) uses robust fingerprints, it may not be sufficient against many automated avoidance techs one could throw together quickly

# orderedItems[].object.content
It just seems that the combo of text being directly human editable sets it apart from video and images, in addition to the fact that writing style is so wildly diverse... I just don't see a world where we reach an acceptable level of accuracy with any reasonable approach

# orderedItems[].object.contentMap.en
It just seems that the combo of text being directly human editable sets it apart from video and images, in addition to the fact that writing style is so wildly diverse... I just don't see a world where we reach an acceptable level of accuracy with any reasonable approach

# orderedItems[].object.content
Am I the only one who thinks LLM text detection (e.g., ChatGPT) is an unsolvable problem long-term? Zero-effort human edits or basic synonym replacement would defeat almost anything we can come up with (fingerprinting, word likelihood, etc.).

#chatgpt #openai #nlp #nlproc #ai

# orderedItems[].object.contentMap.en
Am I the only one who thinks LLM text detection (e.g., ChatGPT) is an unsolvable problem long-term? Zero-effort human edits or basic synonym replacement would defeat almost anything we can come up with (fingerprinting, word likelihood, etc.).

#chatgpt #openai #nlp #nlproc #ai

# orderedItems[].object.content
@oscarkjell Yes totally, definitely a super close cousin of confirmation bias and seems to fall under the bigger umbrella of motivated reasoning. Appreciate the insight!

# orderedItems[].object.contentMap.en
@oscarkjell Yes totally, definitely a super close cousin of confirmation bias and seems to fall under the bigger umbrella of motivated reasoning. Appreciate the insight!

# orderedItems[].object.content
@ktaylor Ahh thank you!

# orderedItems[].object.contentMap.en
@ktaylor Ahh thank you!

# orderedItems[].object.content
@CircleReader this is PERFECT, and thanks so much for the references!

# orderedItems[].object.contentMap.en
@CircleReader this is PERFECT, and thanks so much for the references!

# orderedItems[].object.content
Is there a formal name for the phenomenon where people seem to apply greater levels of scrutiny or criticism to stats/reports/research they disagree with than those they agree with?

I've been calling this "differential scrutiny" but I feel like there has to be a more common name

#science #scicomm #econtwitter

# orderedItems[].object.contentMap.en
Is there a formal name for the phenomenon where people seem to apply greater levels of scrutiny or criticism to stats/reports/research they disagree with than those they agree with?

I've been calling this "differential scrutiny" but I feel like there has to be a more common name

#science #scicomm #econtwitter

# orderedItems[].object.content
(And while this slice isn't perfect -- much more investigation to do -- no evidence of algorithmic bias! If anything, alignment with human coders is *stronger* when examining data from underserved groups. Could reflect that it just aligns with existing human biases, though this human coder group seems unlikely to exhibit this issue)

# orderedItems[].object.contentMap.en
(And while this slice isn't perfect -- much more investigation to do -- no evidence of algorithmic bias! If anything, alignment with human coders is *stronger* when examining data from underserved groups. Could reflect that it just aligns with existing human biases, though this human coder group seems unlikely to exhibit this issue)

# orderedItems[].object.content
Been doing some work to validate an NLP algorithm's output against a team of human coders for a project, and it is REALLY hard to overstate how satisfying this one plot is.

Can't give much detail now, but the takeaway is: it was basically *not possible* for it do a better job.

#nlp #data #datascience

# orderedItems[].object.contentMap.en
Been doing some work to validate an NLP algorithm's output against a team of human coders for a project, and it is REALLY hard to overstate how satisfying this one plot is.

Can't give much detail now, but the takeaway is: it was basically *not possible* for it do a better job.

#nlp #data #datascience

# orderedItems[].object.content
@paulgp Ahh amazing, I'll dig in today!

# orderedItems[].object.contentMap.en
@paulgp Ahh amazing, I'll dig in today!

# orderedItems[].object.content
Has anyone with more experience on Mastodon made an R-for-Stata-users style doc explaining some of the idiosyncrasies in switching over from the academic's perspective? @osc + @paulgp seem well poised but obvi don't want to obligate anyone

# orderedItems[].object.contentMap.en
Has anyone with more experience on Mastodon made an R-for-Stata-users style doc explaining some of the idiosyncrasies in switching over from the academic's perspective? @osc + @paulgp seem well poised but obvi don't want to obligate anyone

# orderedItems[].object.content
@AndrewCharlesBaker nature is healing

# orderedItems[].object.contentMap.en
@AndrewCharlesBaker nature is healing

# orderedItems[].object.content
@osc Mmm interesting... My toots will still show up in the on-going local and federated feeds though, right? Though I guess to your point that's a bit too transient/chaotic a thing to rely on for discoverability in the long run

# orderedItems[].object.contentMap.en
@osc Mmm interesting... My toots will still show up in the on-going local and federated feeds though, right? Though I guess to your point that's a bit too transient/chaotic a thing to rely on for discoverability in the long run