Research Matters

Written by John Abowd, associate director, Research and Methodology Directorate

Today, the U.S. Census Bureau awarded two cooperative agreements to research teams at Georgetown University and Purdue University. These teams of university-based researchers are at the forefront of the emerging field of privacy-preserving data analysis, and their efforts will assist the Census Bureau in ensuring we continue to be a leader in protecting confidential information.

The Georgetown University project will help develop methods for publishing data that satisfy both formal mathematical privacy requirements and legal standards for privacy protection. Their research, combined with ongoing research at the Census Bureau, will provide improvements to existing methods that protect privacy by avoiding the release of any information that would identify an individual or business in public statistics.

The projects complement new initiatives within the Census Bureau to strengthen our disclosure avoidance methods, especially as they apply to the detailed publications that result from our flagship products: the 2020 Census, the American Community Survey, and the 2017 Economic Census.

The team from Georgetown University, led by Kobbi Nissim, includes two of the computer scientists who originally developed the theory of differential privacy — the first privacy-preserving data analysis model — as well as leading researchers from Harvard University who specialize in cryptography and information law. Their work for the Census Bureau will help improve the way we understand and implement our statutory mandate to protect the confidentiality of all respondent information in the Big Data era.

The Purdue University project will investigate methods to improve the usefulness of anonymized data by studying systems where automated techniques perform many of the tasks currently performed directly by data analysts preparing the publication products. These private automated techniques have the potential to produce high-quality publishable data without compromising the privacy of the respondents even inside the Census Bureau. This research is complementary to our ongoing research on methods that strengthen traditional disclosure avoidance techniques.

Chris Clifton, a computer scientist with an extensive research record in data anonymization leads the team from Purdue University. He is a past program director in the National Science Foundation’s Computing and Information Science Directorate. Their research is expected to help the Census Bureau better understand how to preserve the suitability of our data products for their many uses once we adopt modern privacy-preserving anonymization methods better adapted to the Big Data era.

Both awards are three-year collaborative efforts that will provide us with the time to research, test and further refine innovation methods to enhance our assurance of the protection of confidentiality mandated by U.S.C. Title 13.

The Census Bureau’s mission is to serve as the leading source of quality data about the nation’s people and economy. We honor privacy, protect confidentiality, share our expertise globally, and conduct our work openly. These new cooperative agreements provide complementary approaches to innovative methods and procedures for executing the dual statutory mandates in Title 13 U.S.C. — collect data in order to publish statistics and maintain the confidentiality of respondent information.

Moving forward, the Census Bureau intends to use Cooperative Agreement Authority to enter into partnership with leading experts in order to produce innovative work and to ensure that we remain the leading source of quality data about the nation’s people and economy. We will use this important tool to engage with leading experts in academia, researchers and nonprofit agencies. Our goal is to find the best sources of data, the best methods to analyze these data, and the best tools to provide data to the public.

By Lucia Foster and Scott Ohlmacher, Center for Economic Studies

Why do some manufacturing plants grow and thrive while others falter? Economists using U.S. Census Bureau plant-level microdata have approached this complex question from at least three different angles. First, they looked at microeconomic patterns at the plant level. Second, they examined the growth, survival and exit of manufacturing plants throughout the business cycle. Finally, they documented the long-term, secular trends in manufacturing.

Each of these is the subject of recent papers by researchers using Census Bureau microdata from the Annual Survey of Manufactures (ASM), the Census of Manufactures (CM), and the Longitudinal Business Database (LBD). While this blog highlights these papers, extensive literature on this subject using Census Bureau microdata exists (many of these papers can be found in the CES Working Paper series.)

Turning first to the microeconomic patterns, the growth, survival and exit of manufacturing plants depends upon their profitability. Profitability is influenced by many factors internal and external to the plant. One important component of this profitability is the plant’s productivity. Empirical evidence using Census Bureau plant-level data reveals that there are differences in productivity in manufacturing plants even within the same narrowly defined industries. Differences in location and production technology are two possible reasons productivity at manufacturing plants vary, even within the same industry.

The Census Bureau collects information on plant characteristics through the ASM and the CM. However, even when we set controls for relevant characteristics, important differences in productivity remain. According to the productivity literature, Syverson (2011) notes that when using the same measured inputs, a manufacturing plant at the upper end of productivity distribution is able to produce almost twice as much output as a manufacturing plant in the same industry at the lower end of productivity distribution. In discussing possible reasons for these differences, Syverson comments that managers have long been thought to be an important factor, but without data, their importance has been speculative.

The Management and Organizational Practices Survey (MOPS), a supplement to the ASM, is intended to partly fill the data gap by collecting information on these practices. Evidence from the MOPS suggests that management practices are correlated with productivity in manufacturing plants. Bloom et al. (2013) find that “structured” management practices related to monitoring, targeting and incentives are tightly linked to better performance (including higher productivity). These “structured” practices include monitoring a large number of high-frequency key performance indicators (KPIs), setting realistic production targets, making sure that all levels of the organization at the plant are aware of KPIs and targets, and setting bonus, promotion and dismissal incentives based on those targets. While structured management practices are associated with positive outcomes, many plants do not adopt these practices. Researchers are now looking into why there are differences in management practices at plants even within the same firm.

Brynjolfsson and McEhleran (2016) find that the adoption of intensive data-driven decision-making and an increased allocation of decision-making to front-line production workers (versus manager-centric decision-making) is associated with large gains in productivity for plants in industries that are generally capital intensive and utilize “continuous-flow” operations.

The research above focuses on the supply side of profitability, but the demand side is also important. The challenge here is that the Census Bureau does not collect microlevel information on prices. However, researchers have been able to create proxies for the demand side in a limited sample of manufacturing plants for which the Census Bureau collects both revenue and physical output. Using this sample, Foster, Haltiwanger and Syverson (2016) show that much of the growth of plants is dependent on the demand side. They find that new manufacturing plants have higher physical productivity and lower revenue productivity compared to their more mature counterparts, reflecting that new plants set prices low in order to build up their market and grow.

In terms of business cycles, Foster, Grim and Haltiwanger (2016) examine the growth, survival and exit dynamics of manufacturing plants during recent cycles. Regardless of the overall economic conditions, it is generally true that plants that are more productive grow and thrive, while lower productivity plants shrink and exit. In most recent business cycle downturns, this process of reallocation from less productive plants to more productive plants is accelerated. However, they find that in the Great Recession, this reallocation of economic activity from least productive to more productive weakened relative to other downturns. Since this was especially pronounced for young plants, they hypothesize that credit constraints impacted the reallocation. Researchers are also using the MOPS to look at the management and organizational characteristics of manufacturing plants that are able to better weather business cycles.

Finally, researchers have used Census Bureau microdata to better understand long-term trends in manufacturing. Using plant-level data is critical to understanding these trends due to changes in industry classification schemes. Without controlling for these changes, it is unclear what is due to changes in underlying economics activity at plants versus changes in classification.

Pierce and Schott (2016) focus on the decline in U.S. manufacturing employment from 2000 to 2007. They examine the link of this decline to China’s accession to the World Trade Organization (WTO) in 2001. By using the Longitudinal Business Database (a Census Bureau research dataset) and the CM, they examine the response of manufacturing plants while controlling for changes in classification. In addition to changes at the external margin (including within firm relocation of production outside the United States), they find evidence of capital deepening of U.S. manufacturing plants that continued in operation during this period.

One topic that figures in Pierce and Schott’s work is the impact of uncertainty on manufacturing plant’s decisions. They cite anecdotal evidence that uncertainty concerning China’s trade status leading up to its accession to WTO impacted manufacturing plant’s planning decisions. The second wave of the MOPS, which is currently in collection, includes a section on uncertainty. We look forward to research using the MOPS that will enable us to better understand the impact of uncertainty on manufacturing plants’ growth, exit and survival.

Written by: William Wisniewski, Center for Disclosure Avoidance Research

At the U.S. Census Bureau, the Disclosure Review Board is best known as the team that establishes and reviews official Census Bureau disclosure avoidance policies for the public release of data products that do not reveal any information about the survey respondents. Yet, the boards’ members also serve other important and lesser known roles. For example, they work with researchers in the Center for Disclosure Avoidance Research to determine the effectiveness of current disclosure avoidance techniques in protecting data products. In addition, these researchers study and develop new techniques that may be applied to future releases of data products.

This work is critical in meeting the guidelines established under Title 13 and Title 26 of the U.S. Code, which states that the Census Bureau is required to protect the confidentiality of individual respondents when it releases data to the public .

This seemly simple mission can often pose challenges. For example, what occurs if a researcher wants to release counts and demographic characteristics of individuals in every county in the United States? What if a researcher wants to release an infinite number of variables in a Public Use File? What should a researcher do if they encounter small cell sizes within their data product?

These types of questions and others, along with their solutions, will be presented in a topic-contributed session at the 2016 Joint Statistical Meetings on Wednesday, August 3, 2016, titled ”Innovations in Disclosure Avoidance at the U.S. Census Bureau.” We explain specific issues and walk through some of the methods and techniques that are used to ensure the Disclosure Review Board meets its mission. That is, to support the Data Stewardship Executive Policy Committee in its efforts to ensure that the Census Bureau protects all Title 13 and Title 26 respondent confidentiality of publicly released data products.

Looking to the future, the Disclosure Review Board will also continue to face other challenges. It is likely that Census Bureau and other researchers will need to develop, test, and apply new methodologies and techniques to Census Bureau data, particularly as the quantity of potentially linkable data outside of the Census Bureau increases.

Written by: Andrew Keller and Scott Konicki

When a household does not respond to the census, the U.S. Census Bureau must send a field worker to that address to complete a nonresponse follow-up interview. For the 2010 Census, 72 percent of American households mailed back a completed census form. The remaining 28 percent that did not respond by mail were counted via a census taker that visited their address. In-person interviews are much more costly than getting a response back in the mail. For the 2020 Census, the Census Bureau is researching the possible use of administrative records to provide a status and count for some addresses in the nonresponse follow-up universe—that is, to indicate whether the housing unit is likely to be occupied or vacant, and how many people may live in it. As outlined below, this information will aid in reducing the number of contacts during the nonresponse follow-up operation.

Over the last four years, the Census Bureau has tested various methods using administrative records to reduce the nonresponse follow-up workload. All tests used administrative records modeling with varying levels of complexity. In the tests, the administrative records allow us to split the nonresponse follow-up address universe into three categories: (1) units identified as administrative records occupied, (2) units identified as administrative records vacant, and (3) addresses identified as no determination.

The figure below shows the flowchart of the contact strategy related to administrative records cases for the nonresponse follow-up operation specific to the 2016 Census Test. When administrative records indicated that an addresses was vacant, it received no in-person visits during the nonresponse follow-up operation.

Addresses that the administrative records indicated to be occupied received only one visit in the 2016 Census Test. All units in the nonresponse follow-up address universe, whether the administrative records indicated they were vacant or occupied, did receive an additional postcard by mail during the nonresponse follow-up operation. The postcard told people at these addresses how to self-respond by filling out the questionnaire online or by responding through the questionnaire assistance line. In short, there are several ways before and during nonresponse follow-up that the Census Bureau is attempting to obtain and use self-responses before using administrative records determinations.

The development of possible administrative records models has been guided by comparing models retrospectively against 2010 Census results. Doing so provides a national evaluation of potential administrative records models. However, a difficulty underlying the evaluation of administrative records modeling usage is handling concerns such as undercounts and erroneous enumerations. Although the analysis using the 2010 Census results provides a solid basis for assessing model performance, it is not the only way to measure it.

To learn more about Nonresponse Follow-Up Contact Strategy for Administrative Record Cases, please join us at the Joint Statistical Meetings.

Written by: Brian Dumbacher, Mathematical Statistician, Economic Statistical Methods Division, and Cavan Capps, Big Data Lead, Associate Directorate for Research and Methodology

The Quarterly Summary of State and Local Government Tax Revenue is a sample survey conducted by the U.S. Census Bureau that collects data on tax revenue collections from state and local governments. Much of the data are publicly available on government websites. In fact, instead of responding via questionnaire, some respondents direct survey analysts to their websites to obtain the data. Going directly to websites for those data can reduce respondent burden and aid data review.

It would be useful to have a tool that automatically collects, or scrapes, relevant data from the web. Developing such a tool can be challenging. There are thousands of government websites but very little standardization in terms of structure and publications. A large majority of government publications are in Portable Document Format (PDF), a file type not easily analyzed. Finally, both web and PDF documents have constantly changing formats.

To solve this problem, researchers at the Census Bureau are studying and applying methods for unstructured data, text analytics and machine learning. These methods belong to the realm of “Big Data.” Big Data refers to large and frequently generated datasets representing a variety of structures. As opposed to designed survey data, Big Data are “found” or “organic” data. Typically, these data are created for a click log, a social media blog or an online PDF report, but are innovatively repurposed and used for something else such as inferring behavior. Since the data were not specifically designed to infer, they often have unique challenges.

The goal of this research is to develop a web crawler with machine learning that performs three tasks:

1. Crawls through a government website and discovers all PDFs.
2. Classifies each PDF as containing relevant data on tax revenue collections.
3. Extracts the relevant data, organizes it and stores it in a database.

For task 1, we used the open-source software called Apache Nutch. In a production environment, the process will scale up by distributing the work over many computers and then combining the results.

For task 2, we developed a technique to convert PDF documents to text and re-organize the output. A classifying model applied to the converted PDF determines if the document has relevant data on tax revenue collections. This model uses the occurrence of key sequences of words such as “statistical report” and “sales tax income” and other text analysis techniques.

For task 3, we are considering various ideas. Relevant data would probably be found in tables and in close proximity to key sequences of words. We will explore table identification methods based on the distribution of terminology in the PDF and additional modeling that maps the nonstandard data in PDFs to standard definitions in Census Bureau publications.

The Census Bureau looks forward to continuing this web scraping research and exploring new machine learning algorithms that reduce respondent burden, speed survey processing and improve data collection.

To learn more about the research methods for scraping government tax revenue from the web, please join us at the Joint Statistical Meetings on August 2, 2016.

By: Maggie R. Jones, Center for Administrative Records Research and Applications

Many taxpayers rely on for-profit tax preparation services to file their income taxes. To make tax filing more appealing to taxpayers, preparers offer financial products that speed up the delivery of refunds. However, recent U.S. Census Bureau research suggests that these products may make families less financially secure.

“A Loan by any Other Name: How State Policies Changed Advanced Tax Refund Payments” examines the impact on taxpayers of state-level regulation of refund anticipation loans (RALs). Both refund anticipation loans and refund anticipation checks (RACs) are products offered by tax preparers that provide taxpayers with an earlier refund (in the case of a refund anticipation loan) or a temporary bank account from which tax preparation fees can be deducted (in the case of a refund anticipation check). Each product comes at the cost of high interest rates (often an annual rate of more than 100 percent) and fees and is very costly when compared to the value of the refund.

States have responded to the predatory nature of refund anticipation loans through regulation. The working paper specifically looks at how the implementation of New Jersey’s interest rate cap in 2008 (no more than 60 percent annual rate) of RALs affected taxpayers. Evidence suggests that the use of refund anticipation products among taxpayers living in ZIP codes near New Jersey’s border with another state increased after the policy changed. In other words, New Jersey’s regulation appears to have suppressed the volume of refund anticipation products offered within the state, with taxpayers near the border crossing into a bordering state to use the products.

Meanwhile, border taxpayers’ use of key social programs such as the Supplemental Nutrition Assistance Program, Temporary Assistance to Needy Families and Supplemental Social Security also increased. In other words, after the change in policy, use of both refund anticipation products and social programs increased for taxpayers in New Jersey border ZIP codes compared with other families, indicating greater hardship. The map below shows the ZIP codes used in the analysis.

At one time, the Internal Revenue Service informed preparers if there was an offset on a taxpayer’s refund. Under pressure from consumer advocates, the IRS stopped providing the indicator in 2010. By 2012, all of the major tax preparation companies in the industry had withdrawn from the RAL market, turning to RACs as a replacement. Consumers paid a minimum of $648 million in RAC fees in 2014. The maps show the withdrawal from the RAL market and the increase in the RAC market between 2005 and 2012.

Refund anticipation products pose important questions for policymakers. In order for people filing taxes to receive higher refunds, tax preparers file additional forms that include claims for credits and deductions, which therefore increase tax preparation costs. This translates to higher charges for low-income taxpayers who are eligible for these credits and deductions. Moreover, preparers target RALs and RACs to low-income taxpayers who expect substantial refunds through redistributive credits such as the Earned Income Tax Credit, arguing that RALs and RACs speed up refund receipt and help taxpayers pay off more pressing debt or bills quicker, making low-income families better off. However, some portion of this refund money goes directly from the tax and transfer system to tax preparers rather than intended recipients.