# orderedItems[].object.content
@paulgp Mmm that's a good call, though exactly how much human editing is required to avoid which methods and thresholds of detection is the key Q (juice, squeeze, etc.).

We're also talking an endlessly increasing variety of LLMs, which can be arbitrarily mixed together, and a growing market for LLMs without fingerprinting... So even if a major player (e.g., OpenAI) uses robust fingerprints, it may not be sufficient against many automated avoidance techs one could throw together quickly

# orderedItems[].object.contentMap.en
@paulgp Mmm that's a good call, though exactly how much human editing is required to avoid which methods and thresholds of detection is the key Q (juice, squeeze, etc.).

We're also talking an endlessly increasing variety of LLMs, which can be arbitrarily mixed together, and a growing market for LLMs without fingerprinting... So even if a major player (e.g., OpenAI) uses robust fingerprints, it may not be sufficient against many automated avoidance techs one could throw together quickly

# orderedItems[].object.content
It just seems that the combo of text being directly human editable sets it apart from video and images, in addition to the fact that writing style is so wildly diverse... I just don't see a world where we reach an acceptable level of accuracy with any reasonable approach

# orderedItems[].object.contentMap.en
It just seems that the combo of text being directly human editable sets it apart from video and images, in addition to the fact that writing style is so wildly diverse... I just don't see a world where we reach an acceptable level of accuracy with any reasonable approach

# orderedItems[].object.content
Am I the only one who thinks LLM text detection (e.g., ChatGPT) is an unsolvable problem long-term? Zero-effort human edits or basic synonym replacement would defeat almost anything we can come up with (fingerprinting, word likelihood, etc.).

#chatgpt #openai #nlp #nlproc #ai

# orderedItems[].object.contentMap.en
Am I the only one who thinks LLM text detection (e.g., ChatGPT) is an unsolvable problem long-term? Zero-effort human edits or basic synonym replacement would defeat almost anything we can come up with (fingerprinting, word likelihood, etc.).

#chatgpt #openai #nlp #nlproc #ai

# orderedItems[].object.content
@oscarkjell Yes totally, definitely a super close cousin of confirmation bias and seems to fall under the bigger umbrella of motivated reasoning. Appreciate the insight!

# orderedItems[].object.contentMap.en
@oscarkjell Yes totally, definitely a super close cousin of confirmation bias and seems to fall under the bigger umbrella of motivated reasoning. Appreciate the insight!

# orderedItems[].object.content
@ktaylor Ahh thank you!

# orderedItems[].object.contentMap.en
@ktaylor Ahh thank you!

# orderedItems[].object.content
@CircleReader this is PERFECT, and thanks so much for the references!

# orderedItems[].object.contentMap.en
@CircleReader this is PERFECT, and thanks so much for the references!

# orderedItems[].object.content
Is there a formal name for the phenomenon where people seem to apply greater levels of scrutiny or criticism to stats/reports/research they disagree with than those they agree with?

I've been calling this "differential scrutiny" but I feel like there has to be a more common name

#science #scicomm #econtwitter

# orderedItems[].object.contentMap.en
Is there a formal name for the phenomenon where people seem to apply greater levels of scrutiny or criticism to stats/reports/research they disagree with than those they agree with?

I've been calling this "differential scrutiny" but I feel like there has to be a more common name

#science #scicomm #econtwitter

# orderedItems[].object.content
(And while this slice isn't perfect -- much more investigation to do -- no evidence of algorithmic bias! If anything, alignment with human coders is *stronger* when examining data from underserved groups. Could reflect that it just aligns with existing human biases, though this human coder group seems unlikely to exhibit this issue)

# orderedItems[].object.contentMap.en
(And while this slice isn't perfect -- much more investigation to do -- no evidence of algorithmic bias! If anything, alignment with human coders is *stronger* when examining data from underserved groups. Could reflect that it just aligns with existing human biases, though this human coder group seems unlikely to exhibit this issue)

# orderedItems[].object.content
Been doing some work to validate an NLP algorithm's output against a team of human coders for a project, and it is REALLY hard to overstate how satisfying this one plot is.

Can't give much detail now, but the takeaway is: it was basically *not possible* for it do a better job.

#nlp #data #datascience

# orderedItems[].object.contentMap.en
Been doing some work to validate an NLP algorithm's output against a team of human coders for a project, and it is REALLY hard to overstate how satisfying this one plot is.

Can't give much detail now, but the takeaway is: it was basically *not possible* for it do a better job.

#nlp #data #datascience

# orderedItems[].object.content
@paulgp Ahh amazing, I'll dig in today!

# orderedItems[].object.contentMap.en
@paulgp Ahh amazing, I'll dig in today!

# orderedItems[].object.content
Has anyone with more experience on Mastodon made an R-for-Stata-users style doc explaining some of the idiosyncrasies in switching over from the academic's perspective? @osc + @paulgp seem well poised but obvi don't want to obligate anyone

# orderedItems[].object.contentMap.en
Has anyone with more experience on Mastodon made an R-for-Stata-users style doc explaining some of the idiosyncrasies in switching over from the academic's perspective? @osc + @paulgp seem well poised but obvi don't want to obligate anyone

# orderedItems[].object.content
@AndrewCharlesBaker nature is healing

# orderedItems[].object.contentMap.en
@AndrewCharlesBaker nature is healing

# orderedItems[].object.content
@osc Mmm interesting... My toots will still show up in the on-going local and federated feeds though, right? Though I guess to your point that's a bit too transient/chaotic a thing to rely on for discoverability in the long run

# orderedItems[].object.contentMap.en
@osc Mmm interesting... My toots will still show up in the on-going local and federated feeds though, right? Though I guess to your point that's a bit too transient/chaotic a thing to rely on for discoverability in the long run

# orderedItems[].object.content
(I guess hashtags are important here? Learning with you all... #introduction )

# orderedItems[].object.contentMap.en
(I guess hashtags are important here? Learning with you all... #introduction )

# orderedItems[].object.content
- Work w/ Julie Park examining the hundreds of thousands of open-response descriptions that applicants provide about their extracurricular involvement. Same idea: do applicants use systematically different language to describe their own involvement + accomplishments along lines of race/ethnicity, gender, and SES, even after accounting for other academic qualifications + high school contexts?

Follow to stay in the loop! Excited to learn from y'all and hopefully build community.

# orderedItems[].object.contentMap.en
- Work w/ Julie Park examining the hundreds of thousands of open-response descriptions that applicants provide about their extracurricular involvement. Same idea: do applicants use systematically different language to describe their own involvement + accomplishments along lines of race/ethnicity, gender, and SES, even after accounting for other academic qualifications + high school contexts?

Follow to stay in the loop! Excited to learn from y'all and hopefully build community.

# orderedItems[].object.content
Coming soon?
- Hopefully getting my third dissertation chapter into working paper form! I use the millions of teacher rec letters submitted via the Common App + NLP to examine whether teachers use systematically different language to describe applicants across gender, racial/ethnic, and socioeconomic lines, even after accounting for salient academic and extracurricular characteristics.

# orderedItems[].object.contentMap.en
Coming soon?
- Hopefully getting my third dissertation chapter into working paper form! I use the millions of teacher rec letters submitted via the Common App + NLP to examine whether teachers use systematically different language to describe applicants across gender, racial/ethnic, and socioeconomic lines, even after accounting for salient academic and extracurricular characteristics.

# orderedItems[].object.content
Some recent research you might find interesting: disaggregating trends in college application behavior, individual resources, and academic readiness indicators across detailed race/ethnicity backgrounds --using perhaps the richest longitudinal source of data available on the subject!

Brief one: https://s3.us-west-2.amazonaws.com/ca. research.publish/Research_Briefs_2022/2022_2023_Report_RaceEthnicity_1_2022.10.10.pdf

Brief two: https://s3.us-west-2.amazonaws.com/ca. research.publish/Research_Briefs_2022/2022_2023_Report_RaceEthnicity_2_2022.10.10.pdf

Tweet thread here: https://twitter.com/brhkim/status/1582 016666199695360

# orderedItems[].object.contentMap.en
Some recent research you might find interesting: disaggregating trends in college application behavior, individual resources, and academic readiness indicators across detailed race/ethnicity backgrounds --using perhaps the richest longitudinal source of data available on the subject!

Brief one: https://s3.us-west-2.amazonaws.com/ca. research.publish/Research_Briefs_2022/2022_2023_Report_RaceEthnicity_1_2022.10.10.pdf

Brief two: https://s3.us-west-2.amazonaws.com/ca. research.publish/Research_Briefs_2022/2022_2023_Report_RaceEthnicity_2_2022.10.10.pdf

Tweet thread here: https://twitter.com/brhkim/status/1582 016666199695360

# orderedItems[].object.content
Quick intro! I'm an econ-adjacent education policy PhD who currently works as a data scientist at the Common App, the largest platform for postsecondary applications in the U.S.

I generally explore questions of college access and equity w/ a variety of methods. Follow me for:
(a) deep dives into college apps data
(b) computational social science methods, especially using natural language processing (i.e. text-as-data)
(c) R/Python/Stata ranting and tips
(d) data visualization ranting and tips

# orderedItems[].object.contentMap.en
Quick intro! I'm an econ-adjacent education policy PhD who currently works as a data scientist at the Common App, the largest platform for postsecondary applications in the U.S.

I generally explore questions of college access and equity w/ a variety of methods. Follow me for:
(a) deep dives into college apps data
(b) computational social science methods, especially using natural language processing (i.e. text-as-data)
(c) R/Python/Stata ranting and tips
(d) data visualization ranting and tips