In a week where I am thinking a lot about AI and equity, Google has once again decided to launch an AI overview feature that predictably (ha) has hallucination issues and does not allow users to opt out. Presumably, this also means that all interactions are being used to refine the model.
The egregious one I saw was this attempt to blind folks with dark skin…
According to the summary, it is apparently safe to gaze at the sun for 5 to 15 minutes or up to 30 minutes if you have darker skin. To be clear, it is not safe to stare directly at the sun and the WebMD article does not talk about sun gazing.
What’s interesting about these major issues being encountered is some of the stories of success coming from civic tech (see previous brag about knowing Claudie). This Journalism Professor Made a NYC Chatbot in Minutes. It Actually Worked – The Markup:
Colin: Do you have any idea why your version worked, when the city’s seemed to fail so badly?
Soma: If I were being gracious, I would say that there is always the ability of the AI to make things up and hallucinate. Additionally, if you have a very large set of documents, it might be difficult to find the ones that are actually relevant. Because that’s an automated process, and maybe when I say, “Can I take a portion of my workers’ tips?” it finds documents that might not be relevant. And then the chatbot isn’t able to answer because the documents aren’t necessarily relevant.
But there were so many basic, basic questions that were answered incorrectly, I can’t imagine the documents were actually loaded. There must just have been gross malfeasance in terms of setting this chatbot up. Because it’s not hard to have it be right most of the time. The issue was getting it to be right 100 percent of the time. And from what I can tell, this city-based chatbot was almost never right, which is hard to do. It’s hard to be that wrong.
This week I’m going to be attending the Accessible Canada Accessible World Conference virtually. I’m also supporting the logistics for a talk on Ensuring Accessible and Equitable Artificial Intelligence by the Director of the OCAD Inclusive Design Centre next week. So I’ve been knee deep in a bit of an inclusive design rabbit hole, and specifically in Dr. Treviranus’ work and where it challenges and interacts with policy around accessibility and equity in AI and technology.
The most directly relevant to the Google AI Overview story is this piece Big Ideas: If you are unique, numbers are not your friends – The Michener Blog. In this post, Dr. Treviranus identifies that the use of statistical thinking to underpin the design is inherently always going to have biases:
“What happens if you are a small number? If the value you create is not quantifiable? If what you want or need to write or talk about is not popular? If the government program that is essential to you does not benefit many other people? If the products and services you depend on only have a tiny market? You will likely discover that you do not count or you do not measure up.”
What’s the alternative then to designing on this basis?
This is covered in Inclusive Design: The Bell Curve, the Starburst and the Virtuous Tornado - Inclusive Design Research Centre (ocadu.ca). You have probably seen the bell curve before. In statistics, the central limit theorem is when variable (ie. people’s needs) are collected and plotted the larger the random sample size, the closer it will look to a normal distribution where 80% of the values are clustered around the mean (average) and the remaining 20% trailing on either side. This became the 80/20 rule which was applied to business by Richard Koch who advised designers to ignore the “difficult 20%” (that take up 80% of the space and effort) and to focus on the sweet spot in the center.
The solution proposed by this article is: rather than designing for “most people”, the aim is to design for an increasing amount of people constantly. By looking at those who have difficulty using your design or can’t use your design at all and building in refinement, you are likely to land on something that also more resilient and adaptable given that humans, the environment, and the needs are not static.
The 80% of the space and effort outside the conventional middle is a vast and shifting terrain, that cannot be reached in a straight line. It is to be approached with humility and without the hubris of predetermined assumptions and presumptions. We do not venture here armed with a hypothesis. Here a linear logic model does not work.
We use the Virtuous Tornado as our planning tool, iterating further and further to the edge, constantly re-calibrating and evaluating our progress, and asking “who are we missing” to recruit the necessary knowledge and lived experience we need.
More on the Virtuous Tornado visualized in the image above can be found in the Inclusive Design Guide from OCAD.
Technoableism
In reading another one of Dr. Jutta’s posts, ReCount 7: AI’s Collateral Damage** | by Jutta Treviranus | Medium, this part really struck me:
Beyond the direct harms of things like automated decision systems, the most liberating technologies are not financially accessible or usable by the people that need them the most. The existence of these technologies gives the false impression that the barriers can be dismissed or ignored)
The blog post that was linked to in this section gave me a lot to think about in terms of the relationship between tech and Disability, How Innovation Sets Me Backwards. Tech that could be enabling me is… | by Aubrie Lee | Immerse (We Count Recount - Inclusive Design Research Centre (ocadu.ca).
It made me think of the idea of cyborgs and what makes us human. There are a lot of items that as a neurodivergent person, I would consider critical adaptive tech. However, there are many spaces where it may be unwelcome, namely stim toy/doodling/knitting and other self-regulatory movements which others may find distracting. So in fact, where ever possible I choose to join long meetings and learnings virtually even where there’s an in person option as I need to be able to move around and not be self-conscious to be able to be fully present. In a way, although it is not a prosthetic arm, being asked to leave them behind does feel like going out of the house without a critical piece of myself. Not sure what the point is, but looking at adaptive devices and tools as part of oneself does feel at odds with some of the issues I have seeing with colleagues having difficulty getting accommodations approved in light of the RTO policies….
It seemed timely with all of that to revisited a book I’ve been chewing on for a few months now, Against Technoableism by Ashley Shew. Here is also a link to her website if you want to get a sense of her and her work first.
Future of Data in the GC
Hey hi. Speaking of cyborgs, this is the speculative fiction section of this week’s note. I’m lucky that I get to talk to so many people who intersect with data work in the GC in some way. I spend a lot of time between conversations reflecting on what we discussed to look for trends and weak signals. So I was asked by two separate people this question about where the gaps are and where they should focus their learning. So here’s my current answer, I’d love to know if you disagree and what you’re seeing from your side of the world!
In terms of specific tools, I think PowerBI is worth learning as it's part of the M365 suite and so is emerging as a common tool across the GC and in other industries, although it does not offer much flexibility or more deep data science work (but this is very rarely what organizations need).
I would say the bigger gap we're seeing is actually in the ability to support senior management in making decisions with data once the dashboard/report/visualization has been made and the ability to support policy making with data. Lots of folks coming out of school now with coding skills, but the communication and policy making skills and bridging between non-technical/technical people are often missing.
I would say learn coding if you are interested in it regardless of your job interests, but at the moment there's not a lot of consistency in terms of what program is allowed to be used at the departments or across industries. I'm hearing a lot of frustration from data scientist who were hired to code, but see their skills decline because they never get to actually use it. In many cases, they end up spending more time briefing, coaching others around them on data literacy, wearing "all the hats" when it comes to data.
As well, there are many hiring managers who are under pressure to increase the data capacity in their organizations, but may lack the data literacy to properly assess a data candidate’s skills. This results in frustration or risk aversion in the hiring process. Having a common language to talk about data skills is a gap is currently being filled ad hoc by candidates who are able to articulate their skills and experiences effectively, or hiring managers relying on poaching from one another, or having working level employees step outside their comfort zones into strategic HR functions to various level of support and experience and outcome.
Upcoming events that look interesting
For those with access to CSPS, join us for "Ensuring Accessible and Equitable Artificial Intelligence" with Dr Jutta Treviranus, PhD, Director of the Inclusive Design Research Centre at Ontario College of Art and Design University, who will present her research and share her insights into what AI reveals and what it obscures, especially for people with disabilities. Following the presentation, there will be a panel on these considerations in a GC context.
In celebration of the Summer Solstice, I would love to spend some time together at this recently open exhibit, Radical Stitch at theNational Gallery of Canada, from 5pm to 8pm on June 20, 2024. Are you an employee of the GC interested in exploring Data through Art and want to go on a field trip followed by a social at the Tavern at the Gallery? Get in touch for the invite.
Links that I’ll try to remember to paste in every week if I remember
Connect with over 4,000 data people to share resources and job opportunities in the GC Data - Informal/Unofficial Facebook Group
If these weekly posts are not enough, here are almost 800 links to fill your heart. This is a data resource repository I maintain to keep track of all the things I looked at or relevant to doing data in government. The curation is skewed toward things I am interested in or am actively working on.