Unblocking $50M by creating Meta's first, scalable enforcement system

Creating scalable, compliant content patterns for ad rejections

Glossary

What’s DSA? The EU Digital Services Act (DSA) is a regulatory framework designed to create safer and more transparent online experiences across the European Union. It applies to digital platforms that host or distribute content, goods, or advertisements. One of its core principles is transparency in advertising and content moderation. This means platforms can no longer give vague or generic explanations for ad rejections.

What are Advertising Standards? What you can and cannot advertise on Meta is governed by our Advertising Standards. These cover over 35 different policies and govern our rules around products or content like guns, nudity and cosmetic surgery. They touch upon some of the most culturally-sensitive, heavily-litigated problems and products of the moment.

Problems

Business

Some 15 million ads are rejected every single day, resulting in $246M/year in lost revenue.

In 2020, the EU announced new regulatory requirements around "transparency" as part of the Digital Services Act, which required changes to how we notified users about enforcements, requiring us to build a design system for how we communicated these issues.

Because we were shipping to multiple surfaces across FOA (family of apps, e.g. Facebook, Instagram) and our advertising surfaces (Ads Manager, Business Support Home), it needed to be scalable.

User

We fire on average 90k emails every hour to inform advertisers about issues with their content. While, some ads should rightly be taken down and remain rejected, 50% of all ad rejections are remediable, which is to say the ad could be fixed by the user taking a simple action such as tweaking the creative content slightly to follow our rules, or appealing. However only 2% of advertisers were doing this. How might we help advertisers understand and resolve issues?

What I did

Who I worked with

3 Product Designers 2 Content Designers 1 UXR 5 Engineers 1 PM 30 Policy Managers 15 Ops Managers 1 TPM

Constraints

How I did it

⚽️ Step 1. Kick off

Understanding users

We started by digging into user sentiment (CSAT) and a literature review around content rejections.

Here’s what we found:

This was followed by a research trip to Brazil to interview advertisers in person. We learned that the lack of transparency for why their ad was rejected, with no clear recourse to resolving the issue was leading to deep despondency, including advertisers feeling:

Meanwhile, poor transparency and explainability was kicking off a flywheel of issues elsewhere. The ad rejection experience had completely eroded user trust in our content moderation process with users increasingly disregarding our reasoning for rejection entirely; choosing to duplicate flagrantly bad content, which in turn was leading to further (more severe) enforcements on their ad account and eventually across their broader business.

This told us that we had a lot of work to do in terms of (1) driving clarity; (2) returning accessible and clear resolution pathways and (3) demystifying the enforcement process.

🔎Step 2. Mapping what existed today

We then set about auditing the existing experience. This proved to be extremely challenging. To trigger an enforcement, of course, you need to be enforced upon yourself. At the time, the team had no easy way of doing this, so we had to create our own ad accounts and publish violating content. There were no prior design files for this work, so to see the full remit of enforcement messaging, I had to search through thousands of lines and lines of code. Some of this lived in a decentralised CMS, rather than our central codebase, so it couldn’t be mapped at all. In short, we were starting from scratch.

This is one message that was returned for Profanity policy on Ads Manager; our main advertising and support surface.

Issues:

Our team does not own any of the advertising surfaces where we ship these experiences. These are owned by the all-powerful Monetization org, who have very rigid pre-determined design systems that we must plug our own content into. To change even a single comma requires extensive review with surface owner teams that can take months.

✍️Step 3. Creating the structure

Aside from low trust in our enforcement systems, user research told us advertisers had high apathy for taking any resolution action that was ‘time-consuming’. This was a challenge for us because 50% of all enforcements were entirely remediable; that is to say, could be resolved by doing a simple action, such as verifying their account or updating audience age. What techniques could we employ to drive engagement and make the process as pain-free as possible?

We started by thinking about what the ideal experience should look and sound like.

👀 For look, we developed these design principles:

🎵 For sound, we applied the Santa Clara principles. This used the notion that transparency builds trust and trust drives engagement, as the cornerstone of all good user experience. It said that all users need to know:

Initial proposal

Following these principles, we mapped out what the IA should look like. Users needed to know ‘what’ and ‘why’ to justify the ‘resolution’, thus the IA followed least to most tangible information.

Each of these principles mapped to a different Article in the Digital Services Act and meant we could demonstrate compliance should we be audited by the European Union.

We called this framework GAME (Governance, Accountability, Measurable, Experiences). We created documentation that was published on our Meta-wide centralised Content Design Wiki so that all design teams who were working on integrity experience should follow the same principles. It’s still in use today and had become a standard template for communicating enforcements.

🪟Step 4. Defining transparency

The next stage was layering on the risk element. Not all policies are created equal and some carried greater risk of circumvention (bad actors abusing our system to publish low quality or illegal content); or complexities to do with local regulation or cultural sensitivities. This meant when I was approaching the messaging side, I needed to work hand-in-hand with policy to establish what we could and could not tell users.

To do that, we developed a transparency framework.

If policies fell into the ‘high risk’ bucket, we would return something called, basic transparency. For everything else we returned high transparency.

Basic transparency was reserved for extreme cases; where we don’t know the reason for the violation, or there’s a criminal investigation where any additional transparency might alert them. This was also our ‘red cord’ for escalations.

High transparency was our default experience for 99.9% of cases. Here we would give the advertiser:

Final designs

GAME on Ads Manager

GAME on Business Support Home

Impact