Leaving slide mode.

Better Evaluations for Everyone: What’s New in Updated WCAG-EM

Jeroen Hulscher, Hidde de Vries (Logius)

CSUN, 11 March 2026. Anaheim, United States

Slides: hidde.blog/slides-csun

I'm Hidde

view of rotterdam skyline with lots of trees visible

Comparing accessibility

Our context

Universal Declaration
of Human Rights

Adopted in 1948 by United Nations General Assembly.

Enshrines basic rights and fundamental freedoms of all human beings

I promise this is not a long history lesson, but I'd like to go back briefly to where this comes from, in our context. A lot of where we are now has its roots in the Universal Declaration of Human Rights, adopted in 1948 by UN General Assembly.

UN Convention on the Rights of Persons with Disabilities

2007: signed on behalf of the European Community

2011: entered into force in the European Union

legally binding, sets minimum standards

Of course all human beings includes human beings with disabilities, but it wasn't until more than half a century later that the UN created the Convention on the Rights of Persons with Disabilities. It was signed on behalf of EU countries in 2007 and got into force in 2011.

It is legally binding and sets minimum standards, European Commission reports back to UN regularly

map of the world that shows that almost all countries in the world ratified the un convention, a few countries are lighter blue including the US, meaning signatory.

It is ratified in most places, including Europe, where we are based.

European regulation to improve digital accessibility

One part of the implementation of UNCRPD was to introduce regulation, WAD and EAA.

Logius + accessibility

Our department prepares for regulatory oversight (in context of WAD), produces tools to help with it, contributes actively to standards, and brings together digital accessibility expertise.

And that brings me back to my organisation and why I'm here… it explains why we're interested in WCAG and specifically evaluating WCAG methodogically.

~300 organisations*,
~9000 websites and apps

that we want to understand the accessibility of.

*cities, provinces, water boards, agencies, national government departments, etc

And that brings me back to my organisation and why I'm here… it explains why we're interested in WCAG and specifically evaluating WCAG methodologically.

How many websites in an org?

screenshot of rotterdam.nl

I live in Rotterdam, The Netherlands, and as a city, that is one of the many hundreds of organisations that fall under the WAD. For illustration purposes, this is not a real list of their websites, just to show the type of websites they may have.

This is their own website

screenshot of afvalwijzer

this one is about rubbish collection, when they come to collect. May be a SaaS tool

screenshot of afvalwijzer

this one is lost & found items, which in NL is a responsibility of city governments to deal with… long story… 

screenshot of city archives

this is the city archive's page… they hired an agency as they didnt feel their work would belong on the main city website. 

screenshot of visit rotterdam

this is the visit Rotterdam website, they really want people to come visit and hired a marketing agency to do this one. It runs on a funky new CMS.

screenshot of meeting notes website

this is the website with all the meeting notes of the city council's meetings. There's a couple of vendors in the country that provides systems for it, they picked one.

photo of eurovision branding

I'm not sure if you're familiar with Eurovision Song Contest here? We hosted it a couple of years ago; of course it needed a website too. A temporary one, flashy… more budget for shiny details than for accessibility

photo of eurovision branding

I could go on and on… of course there's a jobs website, if you want to work for the city this has the job ads and pictures of people hard at work. Different agency, connects to the HR system…

app store screenshot

Last example… well… there's an app to report stuff, ReportR… 

City of Rotterdam

(example of websites an org may be responsible for, illustration purposes only)

So all in all there's a lot of properties. I looked it up and they actually have 122 websites and apps. They'll have different CMSes, product managers… some may be done by an agency, others by internal teams, others directly bought of the shelf from a supplier.

If these are the sites they are responsible for, and again, this is fictional just to illustrate, they would have to show a report based on WCAG-EM to say what this is.

If you're responsible for the accessibility of many websites, you need an evaluation method

What is WCAG-EM?

A brief intro

A standard to comply with regulation: EN 301 549

Points to WCAG for most requirements on websites (and more)

What happens in Europe with regulation is that the European Commission will request standards that can bring “presumption of conformity”. If your product conforms to the standard, it can be presumed it complies with the regulation. They are voluntary though, you are free to to whatever else, but it will be harder for you to show you comply.

EN 301 549, version 3.2.1, is that standard for WAD, version 4.1.1 will be that standard for EAA.

We want to understand how well websites/apps “meet WCAG”

(and yes, conformance is “web page”-based)

screenshot of wcag-em 1 spec
screenshot of wcag-em 2 spec

We are updating to version 2.0, the examples in this presentation will use the wording from 2.0

WCAG-EM

WCAG-EM

Workflow diagram that depicts five sequential steps: 1. Define the evaluation scope; 2. Explore the target website; 3. Select a representative sample; 4. Audit the selected sample and 5. Report the findings. Each step has an arrow to the next step, and arrows back to all prior steps. This illustrates how evaluators proceed from one step to the next, and may return to any preceding step in the process as new information is revealed to them during the evaluation process.

Step 1: define the evaluation scope

Step 2: explore the target product

Step 3: select a representative sample

Sample size depends on product size, age, complexity, consistency.

Step 4: evaluate the selected sample set

Step 5: report the evaluation findings

What WCAG-EM gives us

a way to report on the accessibility of a whole website
(especially if it's too big to look at everything)

Why WCAG-EM helps us monitor many websites at once

for us that comparability is helpful, as the Dutch government, like all EU Member States, needs to report yearly on the accessibility of all government websites (8000+ domains we know off)

City of Rotterdam

(example of websites an org may be responsible for, illustration purposes only)

So to return to what I talked about earlier… Dutch public sector bodies are reporting with these kinds of reports, they'll have one for each of websites and apps.

screenshot of digital accessibility reports dashboard

we even have a public dashboard that shows the data we collect. The input is WCAG-EM based reports.

screenshot explained in comment

Organisations (and everyone in the public) can see how many domains they're responsible for. This is an example of our own org, because yeah, we're also a public sector body. We own 65 websites and apps. They all comply wiht the law, as they have A, B or C status. The A means fully meets WCAG, B means it was evaluated and fixes are known / being worked on, C means an evaluation was planned.

screenshot explained in comment

It shows how we did over time, with bar charts per month in months that had changes. In 2024 we still had a few websites and apps that we didn't look into.

screenshot explained in comment

It also has a way to compare how this organisation is doing compared to other organisations of the same type. So cities can compare with other cities, provinces with other provinces, and we, an agency, with other agencies.

screenshot explained in comment

There is a long list of results, it shows how many WCAG criteria were met per website or app, there are also CSVs available.

WCAG-EM 2.0

Currently published as Draft Group Note, soon to be Group Note

WCAG-EM 2.0

Worked on in the Accessibility Guidelines Working Group (AGWG) for last ~7 months, editors Steve Faulkner (Tetralogical), Hidde de Vries (Logius), Jeroen Hulscher (Logius).

WCAG-EM 2.0

WCAG-EM 2.0: broader scope

“web site” → “digital product”

Examples of digital products include websites, apps, kiosks…

WCAG-EM 2.0: broader scope

“sample” → “sample set”

set of screenshots

WCAG-EM 2.0: broader scope

“web page” → “view”, “sample”

Examples of types of views include web pages, screens of apps, screens of kiosks.

WCAG-EM 2.0: status

Published as Draft Note, open to feedback

Group Note in the next few months.

Thanks! Any questions?

Slides + links on hidde.blog/slides-csun

Contact via hidde.blog/contact