Case Illustration: Small-N Analysis

Understanding Policies (10700193-90)

Yue Hu

Tsinghua University

Administrative

Overview

• WHAT is small-N analysis (SNA)?
  • Defining it by sample size
  • Five common approaches
• HOW does SNA differ from large-N analysis (LNA)?
• WHAT makes a good SNA?

SNA: An Introduction

Starting with Sample Size

Large N

100 (LLN), > 1000 (CLT)

Small N

N ≝ 1~100

How about 0 N (😮)

Check out the Gaming Political Science, GPS

Common Approaches of SNA

Five paths—different lenses, one shared move:
Meaning before measurement.

1. Value judging (critical theory)
2. Story telling
3. Narrative analysis
4. Dramaturgical analysis
5. Category analysis

Value Judging

Whose values shape the policy story?

• Identify the “communities of meaning” around an issue
• Examine how their framings and interactions construct political reality

Lakoff et al. (2004)

Graham Tillett Allison Jr., American political scientist and professor at the John F. Kennedy School of Government at Harvard, in his book of Essence of decision, he attributed the missile crisis of Cuba to Russia’s ideological flaws.

Another example: George Lakoff: Don’t Think of an Elephant!

Writing about framing and metaphor, a book writing for liberal peoples. Thus called them (Democrats) the 进步的, the Republicans 保守派

Story Telling

Render a group, place, or policy through deep description—built from archives, interviews, and immersion.

Balay (2018)

Winner of the Lambda Literary Emerging Writers Award. PhD students at University of Michigan’s American Culture.

我想以“附近”（the nearby）作为研究框架，去深度描绘这些在我们日常附近中出现的打工者，他们是帮我们找房子的中介、为我们做家政的阿姨、给我们送外卖的外卖员……我想通过这本书来记录这些陌生人的状态……他们和我们之间会有什么样的交集，这样一种交集也是跟“真实的附近”产生亲密的关系…… (严飞 2022)

Huang (1981)

Another example: 1587, a Year of No Significance: The Ming Dynasty in Decline，一个皇帝和五个名臣的故事, 萬曆十五年, 黃仁宇

Narrative Analysis

Skocpol (1979)

• Going beyond telling a story
  • Using story to make an argument
  • Putting an argument in the thick context

An outstanding example of comparative history analysis

Dramaturgical Analysis

body

How do policy and policy makers “act”? “Actors,”“Content,”“Theme,”“Space,” etc.

The state is not the reality which stands behind the mask of political practice. It is itself the mask which prevents our seeing political practice as it is. (Abrams 1988)

A sociological approach

Dramaturgical analysis is a way of looking at our everyday life as if we were actors on a stage in a play. Imagine you’re in a school play and you have different scenes where you act as different characters — maybe you’re a student in one scene, a friend in another, or a sibling at home in another scene.

In each scene, you behave differently depending on who is watching you and what your role is. When you’re with your teachers, you might act polite and focused. When you’re with your friends, you might be more relaxed and playful. And at home, you might act a little differently around your family.

The idea comes from a famous thinker named Erving Goffman. He said that in everyday life, we are all like actors who wear different “masks” and perform different “parts” depending on who we’re with and where we are. Just like in a play, we sometimes follow scripts — these are the things we say and do that are expected of us in different situations.

So, dramaturgical analysis is a way to understand how people interact with each other by comparing everyday life to being in a theater play, where we change our behavior based on our roles and the “audience” (or people) around us.

Dramaturgical analysis can be a very interesting way to explore political behaviors and events, as it helps us understand the roles politicians play, how they present themselves, and how they want to be seen by the public. Here are a couple of real-life examples illustrating how this approach can be used to analyze political science questions:

1. Political Campaigns:
   • Situation: During election campaigns, politicians often stage events, speeches, and debates.
   • Analysis: Each politician performs as if on stage, aiming to show their best qualities, win over the audience (voters), and convince them of their capability to lead. The way they dress, talk, and interact are all part of their “performance” to fit the role of a leader. They may change their tone or emphasis depending on whether they are speaking at a rally, a town hall, or in a televised debate.
2. Legislative Debates:
   • Situation: Inside a legislative body, such as a parliament or congress, members debate and discuss various policies.
   • Analysis: Legislators act according to the “script” of their political party or their personal political beliefs. They perform roles that might include being a supporter, a critic, a mediator, or a rebel, depending on the audience and their objectives. Their performances are also for their constituents, ensuring they see their representative fighting for their interests. Legislators’ actions are often tailored to how they will be perceived in the media and by the public.
3. International Summits:
   • Situation: World leaders meet at international summits to discuss global issues like climate change, trade, or security.
   • Analysis: Each leader is there not just as an individual but as a representative of their country. They perform roles that demonstrate their country’s stance and interests. In these settings, leaders are very conscious of the international audience and their domestic audience back home. They carefully choose their words, gestures, and interactions to align with the image and message they want to portray.
4. Crisis Management:
   • Situation: When handling crises, such as natural disasters, health emergencies (like a pandemic), or economic downturns, political leaders must address the public and manage the situation.
   • Analysis: Leaders often adopt a “commander-in-chief” or “guardian” role. They use specific language, attire, and settings that convey control, reassurance, and competence. Their public appearances and speeches are crafted to instill confidence and calm in the public, even if behind the scenes the situation might be chaotic.

These examples show how dramaturgical analysis can provide insights into not just what political figures do, but how and why they perform their roles in the way they do, considering the effect on their audience and their political objectives. This method highlights the theatrical nature of politics, where public perception is as crucial as actual policy-making.

Category Analysis

How do we name the policy world—and what does the name do?

• Identity: welfare recipients, official language…
• Levels: e.g., comprehensive strategic partners, strategic and cooperative partners, strategic partners, constructive cooperative partners… (Strüver 2017)
• Direction: smart city, service-oriented government, neo-urbanization…

• France: 独立自主，与美唱反调；法国成为欧盟唯一常任理事国；法国调查中国新能源汽车，中国报复法国干邑白兰地（送习近平一瓶干邑白兰地）；法欧指责中国俄乌战争，奥运期间停火
• 塞尔维亚：中国台湾对英国塞尔维亚科索沃问题，1999年美国轰炸塞尔维亚贝尔格莱德
• 匈牙利：欧盟成员国，拥抱一带一路

LNA vs. SNA: Different Criteria

SNA rejects four LNA expectations—and replaces each with something else.

Be objective → interpret

Reality is not universal; Policy is constructed.

Be voiceless → speak with

西蒙娜·德·波伏娃

Simone de Beauvoir, The Second Sex, gender is constructed.

One is not born a woman, but becomes one.

Be experimental → embrace context

SNA treats messy impacts and multi-causality as data, not noise.

Be generalized → go deep

Conducting
a Good SNA

Evaluation System

Replacing LNA’s reliability/validity with four trust-building criteria:

• Credibility — can we trust the account?
• Transferability — does it travel to other settings?
• Dependability — was the inquiry rigorous?
• Confirmability — are findings traceable to evidence?

Credibility (汪英伦 2022)

Are the data and interpretations presented in a way the reader can trust?

If the data and implications are presented credibly.

Based on enough description and background

e.g. Censorship, social desirability

Transferability

Can the insights travel to other settings?

当年的SARS也属于冠状病毒，公众容易将二者相类比。当年在应对SARS的过程中，我们是有过教训的。这就提醒我们，千万不可产生轻敌心态和麻痹思想，更不能犯下和当年一样的错误。(王君平 2020)

Built from detailed, illustrative description—but the bridge is built by readers in their own settings, not by the researcher.

The author’s job: make transfer possible. The reader’s job: decide whether it applies.

Dependability & Confirmability

Was the inquiry rigorous (dependability), and are findings traceable to the evidence (confirmability)?

Dependability: a legit process of inquiry

• What kinds of data, over what period?
• Sustained fieldwork, or one visit and a few interviews? …

Censorship

Authoritarianism, communist capitalism can be used to describe nowadays China?

Confirmability: findings clearly linked to analysis, data, and the research site.

There’s an alternative set of criteria which is more philosophical,

Morrow, Susan L. 2005. “Quality and Trustworthiness in Qualitative Research in Counseling Psychology.” Journal of Counseling Psychology 52(2): 250–60.

Criterion Comparison

Different goals, different yardsticks—neither is “better.”

Large-N	Small-N
Objective	Understanding
Voiceless	Voice
Experimental	Comparison
Replicable	Depth
Generalizable	Context

Always care about the framing, and understand an issue from different angles.

• Understanding: the researchers’ background, view, approach, and value; 钟南山更可信
• Voice: Study by Bourgois and Schonberg (2009) of homeless men and women who were addicted to drugs is compelling because it showed they care.
• Comparison: reflexivity: Well-crafted studies provide the reader an opportunity to reflect on his or own situations in regard to similarities and differences.
• Depth: richesness for understanding and context differences
• Context: not only if policy works, but what cases not and why

Take-Home Points

• What’s small-N analysis (SNA)
  • N ≝ [1~100]
  • Common paths
    1. Value judging
    2. Story telling
    3. Narrative analysis
    4. Dramaturgical analysis
    5. Category analysis

• SNA vis-à-vis LNA
  • Role
  • Data collection
  • Analysis
• Good SNA
  • Credibility
  • Transferability
  • Dependability
  • Confirmability

Reference

Abrams, Philip. 1988. “Notes on the Difficulty of Studying the State.” Journal of Historical Sociology 1 (1): 58–89. https://doi.org/10.1111/j.1467-6443.1988.tb00004.x.

Balay, Anne. 2018. Semi Queer: Inside the World of Gay, Trans, and Black Truck Drivers. Illustrated edition. The University of North Carolina Press.

Chock, Phyllis Pease. 1995. “Ambiguity in Policy Discourse: Congressional Talk about Immigration.” Policy Sciences 28 (2): 165–84. https://doi.org/10.1007/BF00999674.

Donohue, John J., III, and Steven D. Levitt. 2001. “The Impact of Legalized Abortion on Crime.” The Quarterly Journal of Economics 116 (2): 379–420. https://doi.org/10.1162/00335530151144050.

Huang, Ray. 1981. 1587, a Year of No Significance: The Ming Dynasty in Decline. First Edition. Yale University Press.

Lakoff, George, Howard Dean, and Don Hazen. 2004. Don’t Think of an Elephant!: Know Your Values and Frame the Debate–the Essential Guide for Progressives. First Edition. Chelsea Green Publishing.

Pader, Ellen-J. 1994. “Spatial Relations and Housing Policy: Regulations That Discriminate Against Mexican-Origin Households.” Journal of Planning Education and Research 13 (2): 119–35. https://doi.org/10.1177/0739456X9401300204.

Schaffer, Frederic Charles. 2000. Democracy in Translation: Understanding Politics in an Unfamiliar Culture. Cornell University Press.

Skocpol, Theda. 1979. States and Social Revolutions: A Comparative Analysis of France, Russia and China. Cambridge University Press.

Strüver, Georg. 2017. “China’s Partnership Diplomacy: International Alignment Based on Interests or Ideology.” The Chinese Journal of International Politics 10 (1): 31–65. https://doi.org/10.1093/cjip/pow015.

严飞. 2022. 悬浮：异乡人的都市生存. 广西师范大学出版社.

汪英伦, dir. 2022. 一场很（没）有必要的春晚. 加拿大.

王君平. 2020. “尽快掐灭疫情传播的‘火种’.” 人民日报客户端 (北京), January 2.

Bonus: More examples

Another Illustration of Dramaturgical Analysis

Commonality and Differences between SNA and LNA

Same

1. Empirical basis
2. Scientific analysis
3. Falsifiability

Different

• Role
• Data collection
• Analysis
• Criteria

Role

• Researchers are human
  • Experiences
  • Cognitive differences
• Researchers participate
  • Observer vs. Insider
  • Undercover vs. Activist

Cognitive differences: men and women; allowing tourists entering the campus

Researchers generalize/create culture, etc.

Researchers have value

Legalizing abortions lowers crime rates across cultural groups and time periods because crime is tightly correlated with unwanted children, and unwanted children with criminality and other social ills (Donohue and Levitt 2001).

“The poor are more likely to commit crimes”
↓
“The poor is the source of crimes”
↓
“Less poor kids, less source of crime”

Poverty Governance

北京市安委会回应“驱赶低端人口”：是在排查隐患

https://baijiahao.baidu.com/s?id=1585068230951214767&wfr=spider&for=pc

Data Collection and Analysis

1. Observing with intention to understand;
2. Interviewing with respondents as conversational partner;
   
3. Reading the documents as they were writing meaningfully;

4. No need to transform words into numbers

   • Numbers are not the results but the sources
   • More details than digits¹
   • Focus:
     • Where did the number or whatever data come from?
     • Where to find the data, what are data, how to know what are saying in the data?…

1. Pader, Ellen-J. 1994. “Spatial Relations and Housing Policy: Regulations That Discriminate Against Mexican-Origin Households.” Journal of Planning Education and Research 13(2): 119–35.

Using American view of occupancy (and thus housing policy enforcement) is discriminated (Pader 1994).

2. Schaffer, Frederic Charles. 2000. Democracy in Translation: Understanding Politics in an Unfamiliar Culture. Cornell University Press.

Interview of people in Senegal about democracy and American democracy (Schaffer 2000)

3. Chock, Phyllis Pease. 1995. “Ambiguity in Policy Discourse: Congressional Talk about Immigration.” Policy Sciences 28(2): 165–84.

By examining the written record of spoken language, she was able to analyse the ways in which legislators’ and expert witnesses’ embedded ideas about immigrants reflected and shaped immigration reform and policy contentions (Chock 1995).

Analysis and collection are intertwined.

1. Narrative reports often read like novels.