Has Sanctuary Personnel Social Work News used generative AI to write its content? — Here’s what we can learn by asking…
Sanctuary Personnel’s Social Work News magazine has a series of articles tagged ‘Social Work Skills’ ‘written and curated by’ the magazine’s ‘content editor’ and offering ‘bite-sized tips to help make you a better social worker’. Concerned that these pieces — which are of questionable quality, largely unreferenced and lacking any real substance — are being presented as valid sources of social work knowledge to students and practitioners in a magazine with no discernible editorial processes and apparently very little in the way of editorial standards, I decided to run them through an AI detection tool. As a social work academic, I am continually looking into generative AI tools that may be legitimately and illegitimately used in the creation of social work writing, as well as in the detection of AI-generated text. I opted to use top-rated AI detector, https://undetectable.ai/, said to have 90% accuracy.
First, a note on method. Each of these pieces is bookended by an intro and a sign off clearly written by the putative author to tailor the content. In each case, I removed these and ran the substantive part (the part actually about the topic under discussion, which makes up the overwhelming bulk of the text) through the detector. (See APPENDIX for control measure.)
Here’s what I found…
Starting with the most recent (at time of writing), WHAT MYERS-BRIGGS PERSONALITY TYPES MAKE THE BEST SOCIAL WORKERS? the result was:
Oh dear!
Now, 5 STEPS TO SUCCESSFUL MULTI-AGENCY WORKING
Result:
Ok, maybe the first was a false positive then…
Next, HOW TO SUPPORT PEOPLE WHO SELF-SABOTAGE
Result: content is detected as written by AI
Oof. OK. Now, let’s look at the rest in turn…
WHAT IS THE DIFFERENCE BETWEEN THE SOCIAL MODEL AND THE MEDICAL MODEL?
Result: content is detected as written by AI
HOW PERSON-CENTRED PRACTICE CAN HELP SAVE THE OFFICE CHRISTMAS PARTY
Result: content appears human
THE MAGIC OF KIDLIN’S LAW AND HOW IT CAN MAKE YOUR PROBLEMS DISAPPEAR (BY HALF)
Result: content is detected as written by AI
HOW YOU CAN USE THE PCS MODEL TO ANALYSE OPPRESSION
Result: content is detected as written by AI
HOW TO DEAL WITH A VEXATIOUS COMPLAINT IN SOCIAL WORK
Result: content is detected as written by AI
HOW TO PREPARE TO TAKE TIME OFF OVER CHRISTMAS
Result: content appears human
HOW TO IDENTIFY AND RESPOND TO COERCIVE CONTROL
Result: content appears human
FIVE TYPES OF POWER AND THE IMPORTANCE OF UNDERSTANDING THESE IN SOCIAL WORK
Result: content is detected as written by AI
HOW TO TELL THE DIFFERENCE BETWEEN SOCIAL WORK THEORIES, MODELS, METHODS AND APPROACHES
Result: content is detected as written by AI
HOW UNDERSTANDING SYSTEMS THEORY CAN MAKE YOU A BETTER SOCIAL WORKER
Result: content is detected as written by AI
HOW TO DEAL WITH A NARCISSISTIC CLIENT (nice bit of labelling there — the piece is an example of why generic ‘bite size guides’ are very often not suitable to the complexities of social work practice)
Result: content appears human
THE MAGIC OF TIMEBOXING AND HOW IT CAN MAKE YOU A MORE PRODUCTIVE SOCIAL WORKER
Result: content is detected as written by AI
THE BEST WAYS TO ENGAGE A NEW CLIENT ON YOUR FIRST HOME VISIT
Result: content is detected as written by AI
HOW TO USE ACTIVE LISTENING IN SOCIAL WORK AND HEAR WHAT PEOPLE ARE REALLY SAYING
Result: content is detected as written by AI
PERSON-CENTRED PRACTICE: WHY IT MATTERS AND HOW YOU CAN USE IT IN SOCIAL WORK
Result: content is detected as written by AI
HOW TO USE MOTIVATIONAL INTERVIEWING IN SOCIAL WORK
Result: content is detected as written by AI
A QUICK GUIDE TO MANAGING DIFFICULT PHONE CALLS IN SOCIAL WORK
Result: content is detected as written by AI
HOW TO BREAK BAD NEWS TO PEOPLE IN SOCIAL WORK
Result: content appears human
HOW TO UNDERTAKE AND RECORD A HOME VISIT IN SOCIAL WORK
Result: content is detected as written by AI
HOW TO WRITE A SOCIAL WORK CHRONOLOGY
Result: content is detected as written by AI
HOW TO BE A MORE RESPECTFUL SOCIAL WORKER (I suggest not treating people like they’re stupid by trying to pass off AI-generated writing as your own would be a good start.)
Result: content is detected as written by AI
HOW TO SPOT DISGUISED COMPLIANCE: THE TOP 5 GIVEAWAYS (This dangerously shoddy, unreferenced ‘guide’ purporting to give handy tips on a highly contested, ethically fraught and complex area lapses into seriously simplistic, suspicion-based reflexes and harmful tropes about the people in highly vulnerable and difficult situations.)
Result: content appears human
A GOOD PRACTICE GUIDE TO WRITING BETTER SOCIAL WORK CASE NOTES
Result: content is detected as written by AI
HOW TO USE THE THREE HOUSES TOOL IN SOCIAL WORK
Result: content is detected as written by AI
TEN WAYS TO BE A MORE PRODUCTIVE SOCIAL WORKER (AND GET HOME ON TIME)
Result: content appears human
So that’s 20 of 28 articles in Sanctuary Personnel Social Work News ‘Social Work Skills’ series that, when run through a top-rated AI detection tool with a vaunted 90% degree of accuracy, are ‘detected as written by AI’.
That’s a hit rate of 71.4%.
You can check these results by running the pieces through the tool yourself.
It always struck me as odd that these ‘Social Work Skills’ pieces are touted as ‘written and curated by’ the author. In light of these results, it occurs that the ‘curated by’ may well be an attempt by the author to pre-empt concerns that AI may have been used to generate these pieces.
If so, and if it is the case AI generated the bulk of these articles, as suggested by the results, ‘curated by’ simply does not wash.
Here’s why:
The use of AI is a new and emerging area of challenge and opportunity in the areas of publishing and social work. Publishing on social work, a highly regulated profession and an academic field, trust in which is founded in high professional and ethical standards, must in particular adhere to high standards of transparency and accountability. If, as is suggested by these results, Sanctuary Personnel Social Work News used AI to write articles framed as practical guides and knowledge resources for social workers and students and not disclosed this to readers, this is at the very least misleading and means the magazine has, once again, gone some way to damaging trust in social work and social workers.
Social workers in England must abide by the standards of the regulator, Social Work England. One of those standards (5.6) states social worker must not:
Use technology, social media or other forms of electronic communication unlawfully, unethically, or in a way that brings the profession into disrepute.
This apparent undisclosed use of AI also raises the question of what else in the magazine may be of questionable provenance, which reflects badly on all contributors. It’s also worth noting some regular contributors are anonymous. Who’s to say they are even real people? As long as they are anonymous, there’s literally no way of knowing. They could be fronts for AI content, or even the confections of content-hungry editors with lax standards… Tangled webs indeed!
The consensus in publishing, including academic publishing, is that any use of AI in creating material for publication should be transparent and open to scrutiny. This article is useful in explaining why full and honest disclosure is crucial. This passage is, I think, usefully instructive:
Authors who employ generative AI in developing papers should transparently disclose their use to editors, reviewers, and readers. Since generative AI is constantly changing and the scholarly community is only beginning to experiment with it, it is not prudent at this time to promulgate hard and fast rules for how generative AI should be disclosed. We recommend, however, that disclosure should describe how the AI was used and should identify AI-generated content. Authors should err on the side of too much transparency rather than too little: when in doubt, disclose. Some ways of disclosing the use of generative AI could include describing the use in a paper’s introduction, methods section, appendix, or supplemental material or citing the generative AI tool in the notes or references. (Kaebnick et al, 2023)
As usual, these things seem beyond the grasp of the editors of Sanctuary Personnel Social Work News, for whom transparency and scrutiny appears to be something they are very keen to avoid. The magazine hasn’t even had an ‘about’ page for several months.
I have asked before: Who is responsible for this magazine’s content? The authors? It’s putative ‘editorial team’? Sanctuary Personnel, which opaquely funds it? It is impossible to tell because there is no readily available information on its current editorial staff, commissioning, submissions or editorial processes. This is, in any analysis, bad publishing practice — even more so for a magazine purporting to champion a profession in which the public’s trust depends on high standards of honesty, integrity and accountability.
Reputable publications, particularly those pertaining to and even carrying the name of the registered profession and academic field of social work, adhere to high editorial standards. Aside from the clearly questionable quality and value of its content, the apparent undisclosed use of AI by Sanctuary Personnel Social Work News to create social work practice guides and knowledge resources is another example of the magazine’s poor editorial standards potentially damaging trust in the profession and field of social work. In doing so Sanctuary Personnel Social Work News not only discredits itself and those associated with it, but also the profession and field of social work and those who work in it.
This is why, as a representative of the profession and someone with professional responsibility for the learning and development of future social workers, I believe it of prime importance to raise these issues.
At a basic level, it is surely unethical for any publication trading on association with the profession, academic field and/or name of social work to allow authors to present work not written by them as their own, or at least without disclosing the use of generative AI to produce the writing.
This is an exceptionally bad example to social workers in time- and resource-poor environments and to students, in particular, who are and will increasingly be encouraged to use AI in their work. In order to maintain and promote accountability, transparency and integrity in the profession, now and in the future, all use of AI in social work writing of any kind should, as a minimum, be fully disclosed and explained by those presenting it.
On this serious, wider point: I note with irony that the tool used here — https://undetectable.ai/ — also has a paid option that allows user to ‘humanize’ AI-generated content, just one of a growing number of tools proliferating across the web as developers rush to capitalise on this emergent, barely understood but rapidly expanding tech. Video tutorials and ‘cheat sheets’ abound, encouraging the use of AI to fabricate university assignments AND pieces for publication. This is of particular concern, especially in the field of social work education, where written assignments make up a large part of assessment as preparation for entry into a profession which is required to uphold high standards of transparency, openness and accountability. It is clear that social work educators and course providers will struggle to keep pace with the rapidly evolving AI tech landscape. If we wish to preserve the integrity of social work education as well as trust in the profession, we should give urgent and serious thought to alternative methods and modes of assessment in which it will be harder to pass off AI-generated material as one’s own. For example, moving away from summarising concepts and demonstrating understanding toward more presentation- and viva-type assessments, and group/collaborative work. Written assessments should emphasise deeper critical and analytical thinking and involve the integration of values and philosophical concerns alongside appreciation of political and ideological factors shaping policy, practice and lived experience.
APPENDIX: Control
For the avoidance of doubt, I ran the text of this very blog through https://undetectable.ai/ and got the following result:
I also ran this piece I wrote on the emerging use of AI in social work practice through the tool and got the same result. And this article, on dissenting social work and immigration. Guess what? Same result. In fact, as a control, I ran several pieces I have written through the detector, always with the same result.
Feel free to do the same.
Thanks for reading!
Kaebnick, G.E., Magnus, D.C., Kao, A. et al. Editors’ statement on the responsible use of generative AI technologies in scholarly journal publishing. Med Health Care and Philos 26, 499–503 (2023). https://doi.org/10.1007/s11019-023-10176-6
Thanks to J. Ritchie for the reminder re relevance of Social Work England’s professional standards to the issue.
