Support all your favorite nonprofits with a single donation.

Donate safely, anonymously & monthly, in any amount. It's a smarter way to give online. Learn more
Sunlight Foundation
Washington, DC

The Sunlight Foundation uses technology and ideas to make government transparent and accountable.

The Sunlight Foundation is:

A think-tank that develops and encourages new policies inside the government to make it more open and transparent.
A campaign to engage citizens in demanding the policies that will open government and hold their elected officials accountable for being transparent.
An investigative organization that uses the data we uncover to demonstrate why we need new policies that free government data.
A grant-giving institution that provides resources to organizations using technology to further our mission and create community
An open source technology community that revolves around the Sunlight Foundation’s core mission

Sunlight Foundation is not verified as a 501(c)3 organization.

Latest News

Oct 31, 2014

Here at the Sunlight Foundation we take Halloween and elections and data very seriously. So it seemed appropriate to rummage through our files and some up with some of the scariest numbers of this year's congressional election. Thanks to Sunlight designer Amy Cesal for this holiday greeting.

spooky house silhouetted against a moon with bats flying under trees

Oct 31, 2014

Adam Roth, founder and CEO of StreamLink Software

The Digital Accountability and Transparency Act (DATA Act), which was passed unanimously in both the House and Senate and signed into law May 2014, is a positive step forward for the open data movement. The law creates standardized data elements, formats and processes, making possible the reporting of aggregated federal spending data. This translates into greater taxpayer visibility, stricter federal funding oversight and smarter resource allocation.

However, the DATA Act foreshadows top-down spending reforms that will require cities and counties to find new and better ways to track and report on federal funds.

The DATA Act rollout timeline

The DATA Act puts the Office of Management and Budget and the Treasury Department jointly in charge of creating standard data requirements for reporting federal dollars. They have one year from the law passing to review current processes and data, and to develop consistent formats and identifiers.

Once complete, government agencies will have two years to apply data standards to information reported to the Treasury, White House and federal grant and contract databases. A pilot program, expanding on the learnings of the Grants Reporting Information Project (GRIP), will test grantee and contractor requirements. However, grantees can voluntarily adopt standards before they are mandated.

Eventually, there may be a single portal for federal award reporting. When this happens, grantees will be able to submit reporting data to multiple government agencies simultaneously.

Photo credit: woodleywonderworks/Flickr

Local government implications and preparation

The DATA Act’s passage means big changes ahead for city and county governments that depend on the $541.9 billion in federal grants awarded in 2014. Municipalities must adopt the infrastructure and processes necessary for compliance with data reporting requirements. Here are some ways that local governments can ready themselves for reforms:

  • Prepare for machine-readable data formats. The DATA Act makes machine-readable formats the standard for federal government data in order to eliminate data entry redundancies. As demonstrated in the GRIP pilot, this will likely be in the form of online web forms, XML single submissions or XML bulk submissions. Grant management software can significantly increase reporting efficiency through bulk or batch XML filing. With it, multiple grant reports can be submitted in a single XML file transfer, saving grant managers time by pulling directly from grant recipients’ existing management systems.
  • Make grant performance outcomes data-driven. Municipalities that are not already reporting quantitative and qualitative outcomes may struggle to secure funding once grantors require measurable results and proof of ROI. Grant managers should assign tasks directly related to grant goals, and track projected versus actual revenue and expenditures to make sure programs stay on course and meet objectives. Regular performance snapshots ensure there are no surprises at the end of the grant cycle.
  • Streamline grant management processes. Under the DATA Act, federal grantees will report the same information as before, but data will be gathered in more efficient ways. That said, aggregated spending data will shine a spotlight on mismanaged funds, upping the competition for already sought-after federal dollars. Local grant managers will need to reduce waste in grant management, reallocating administrative time and resources to more mission-critical program initiatives. Lead recipients have the added responsibility of delegating grant program tasks to sub-recipients and rolling up performance and spending data for consortia grant reporting.

The early mover advantage

As municipalities plan for years ahead, they must invest in the adoption of new processes and technology to support the open data imperative. There is an early-mover advantage.

Compliance will require greater technology and operational infrastructure, such as adequate computing power, data storage and processes to track funds and programs throughout the grant lifecycle. Municipality grant managers will need to improve both program performance and their ability to demonstrate improved performance and measurable outcomes to grantors.

The potential is promising, but the city governments that stand to benefit from spending reforms will be those that pivot their operational, technology and talent infrastructure toward digitized grant management and data-driven award performance measurement.

Adam Roth ( is the founder and CEO of StreamLink Software. The company’s flagship product, AmpliFund, is designed for managing every stage of the grant lifecycle, from pre-award research and planning to post-award performance and reporting.

Interested in writing a guest blog for Sunlight? Email us at

Oct 31, 2014

When we think about using microdata, the individual-level data that permits analysis at the most granular level, we often think about the way its use is restricted by rules protecting individual privacy. Health data, school data and criminal justice records are all restricted due to various privacy concerns. While some of these restrictions seem necessary, others don’t seem realistic or helpful.

Because of that frustration, our exploration into “open data at the power of one” began with a consideration of the state of the art in data anonymization techniques. If anonymization techniques were robust enough that individuals could not be connected with the released data, we could deploy them widely across public microdata-holding sites and feel quite comfortable advocating for the immediate and universal release of microdata. (As our recent post described, however, while techniques are moving in that direction, we’re not quite there yet.)

At the same time, it is also important to point out that not all kinds of individual-level data are held to the same privacy standard. Some data legally contain personally identifiable information. While we wouldn’t have to worry about violating privacy protection laws under conditions of perfect data anonymization, we also don’t have to worry about violating privacy protection laws where individual-level datasets legally contain identified data.

Exploring the varied terrain of legally-identified data helps show how different datasets, even closely related datasets, may have very different levels of protection. Circumstances of history, our legal traditions, and the political ability to fight for privacy or argue a countervailing public interest all affect the legal determination of which personally-identified information can be publicly released and which can not. In other words, the determination of which data can be legally identified occurs through a social and political process and is not an exact science.

While there is no clear or absolute rule for determining which personally-identifiable data should be legally available, arguments about legally identified data always operate by balancing social benefit and individual harm. When there are convincing political arguments that the social benefit strongly outweighs the individual harm, it’s easier to maintain public access to personally identified data. Where there are convincing political arguments that people are significantly harmed by identified data with little countervailing social benefit, policies are more likely to restrict access.

So what kind of data currently does legally include personally-identifiable information?

We can begin to answer that question by considering how the official American collection, provision and restriction of individual-level information has changed over time.

In the colonial era (as now), the right to government-held information sprang from the existence of official collections of data. In colonial America, the mandated collection of such documents as probate records, land-ownership records and vital records provided a core set of public data that individuals could access for specific legal uses. The records of court trials were also available. These, however, were generally available for inspection to anybody, without a specific legal mandate, in line with the ancient common law right to a public trial.

In addition to legal data, public data collection also reflected ongoing public surveillance for social goals, such as monitoring disease prevalence. In his entertaining historical survey of the American effort to balance personal privacy and access to information about others, Robert Ellis Smith describes also how a Puritan belief in the godliness of constant public scrutiny led to the early American public recording of such misdeeds as drunkenness, failing to attend church and living alone (an often-illegal activity.)

Data collection and its public availability changed dramatically in the 20th Century. Daniel Solove describes how the expansion of the bureaucratic state in the 20th Century led to an explosion of public records, far outstripping the scale of individual-level data collection that had occurred to that point. Moreover, while earlier record-keeping law and technology had limited the practical scope of records access, 20th Century technology and political action changed the nature of both data collection and data requests in ways that let both occur far more broadly. Harlan Yu and David Robinson effectively summarize the “closed government” characteristics of public records custodianship in the period up to and after World War II that was countered by the successful effort, led by journalists and media lawyers, to achieve enactment in 1966 of the federal Freedom of Information Act (FOIA). While most states did not have formal freedom of access policy in place before the federal FOIA, all of established similar laws in the years after its implementation.

While FOIA expanded the public right to access government information, it developed simultaneously with a movement to restrict access to government-held individual-level data. The 1974 Privacy Act was a response to fear about increased federal use of collected individual-level data. Expressing concern that the ever-increasing pace of computerized government-held data needed to afford citizens more control over their data, an influential report on citizen rights to privacy in the computer age convinced lawmakers of the need for increased protection. The Privacy Act requires that citizens be able to learn about all of the federal datasets which hold their information, obtain that information, correct faulty information and limit new kinds of uses for the data (except if the new use falls within the twelve permitted exceptions.) While this was unquestionably an important advance in establishing new personal protections, the law applies almost solely to individual-level data housed by federal agencies — although it does place some additional conditions on federal, state and local governments use of Social Security Numbers as identifiers. In response to the same kind of concerns driving enactment of the Privacy Act, many states also developed their own laws to protect citizens’ data. (We will explore the extent of this state-level variation in an upcoming post.)

Significant in its own regard, the Privacy Act was also the first in a series of federal laws which afforded specific kinds of data strong legal restrictions from public release. The Health Insurance Portability and Accountability Act, providing for the confidentiality and privacy of medical records, and the Family Education Rights and Privacy Act, protecting education records, are the two most significant of these federal restrictions on individual-level data release in terms of the sheer number of institutions that must abide by them. However, a number of other specific data confidentiality laws developed similarly in other issue areas, such as the Child Abuse Prevention and Treatment Act protecting records related to child welfare, the Comprehensive Alcohol Abuse and Alcoholism Prevention, Treatment, and Rehabilitation Act setting limitations on access to substance abuse treatment information, and the Gramm-Leach-Bliley Act which limits the release of data associated with individuals’ financial accounts. People seeking to use or access microdata are likely to find that issue-specific laws are ultimately more consequential than the Privacy Act for data access because they address records held by non-federal entities.

The territory of which public, legally-identified data becomes private is constantly changing, frequently subject to pressure from visible public events and their political consequences. For example, the Drivers Privacy Protection Act (DPPA), which protects motor vehicle records, arose after women were stalked and killed using information obtained through their motor vehicle records. (The DPPA changed again after other, less violent public events: A later amendment to this act further restricted state governments from selling motor vehicle record data to marketers in response to a public perception that the practice fueled junk mail.) Another example of the political source of data privacy can be found in the 1988 Video Privacy Protection Act, an expansion of federal law protecting the privacy of individual’s videotape rental records passed in response to the journalistic publication of U.S. Supreme Court nominee Robert Bork’s video rentals. More recently, we’ve witnessed political events drive privacy law in the effort, occurring in a number of states, to restrict public access to lists of individuals who’ve received state gun permits. This nationwide restriction of identified data about gun licensing occurred as a result of a newspaper’s publication of a map of local gun permit holders in the wake of the December 2012 shooting at Sandy Hook Elementary School.

Between the new modes of accessing government-held information provided by federal and state-level FOI laws, and the new restrictions on data created by the Privacy Act and other laws restricting data on specific topics, the landscape of microdata in the early 21st Century is a unique patchwork of highly available and highly protected data. Because of FOIA and advances in electronic access to information, where individual-level data has not been specifically restricted in statute, we can generally find large quantities of individual-level identified data online. Where categories of individual-level data have been legally restricted, it is essentially inaccessible without additional qualifications, certifications or legal intervention.

A survey of a few categories of data demonstrates the highly varied data landscape.

Criminal Justice

  • Maintaining the legacy of common law access, courts often continue to provide fully accessible, individually-identified data. While states may redact elements of court files — commonly including the redaction of information about victims, financial information and information about juveniles — many court records are available to the public via an online interface (albeit for a fee). Observers have noted, however, tension around the availability of court records — and new restrictions on their availability — has arisen where those records contain the kinds of specific identifiers restricted under the issue-specific data restrictions, such as SSNs, drivers license numbers and financial account information, since that information made available on online court records has been the source of identity theft.
  • Police department records — while difficult to access on some topics, notably internal investigations — can be highly identified and available for other subjects, particularly adult arrests. Some states, like Iowa, publish mugshots alongside an individual’s full name, physical details, and previous arrests. States typically more restrictive on privacy, like California, still require police to make public a fairly comprehensive set of details about adult arrestees.
  • Since the 1990s, people convicted of sex crimes have been subject to federal registration requirements that create very highly-identified and specific public data. Sex offender databases are available at the state level online and contain specific identifying details about individuals such as pictures, birth dates and addresses and the crimes they committed, along with distinct social labels like “predator.”
  • Meanwhile, when the criminal justice records concern minors, another set of principles governs the identifiability of data. Court proceedings involving minors have often been confidential in most jurisdictions — though not all — but even where confidentiality is the rule, exceptions are made for very serious and violent crimes.
Political Data
  • State voter registration data contains highly detailed identifying information — including full name, address, political party and recent voting history. It is accessible to some degree in all states (except states like North Dakota, which have no state voter registration) but with varying degrees of access and protection. In Florida, for example, the state makes many personal details publicly available and explicitly acknowledges that users of the Florida voter registration database may use the data in ways that could inconvenience or cost the registered voters. In California, meanwhile, the state makes voter registration information only nominally public by highly restricting access to it, requiring potential data users to demonstrate they are using it for legitimate public interest purposes.
  • Donations to political candidates became a new form of personally identified public data after the 1971 Federal Election Campaign Act (FECA) mandated public disclosure of donations. When the legitimacy of this disclosure requirement was assessed by the U.S. Supreme Court in Buckley v. Valeo (1976), the justices determined that the potential for individual harm caused by the disclosure was outweighed by the social purpose achieved when "disclosure requirements deter actual corruption and avoid the appearance of corruption by exposing large contributions and expenditures to the light of publicity." Significant personally identified details — including name, employer, address, supported candidates and donation amounts — are now available about campaign donors in all 50 states.
  • Even though an individual’s party registration and donation history is public, their votes themselves, however, remain strictly protected. The challenge of producing acceptable online voting had in fact been stymied for quite a while over the problem of being able to adequately demonstrate that the method would secure the anonymity of individual votes.
Public Employee Data
  • The full names, titles and salaries (including benefit information) of people identified as “public employees” are typically considered public information. Many state and local governments put this identified information online as open data.
  • The determination of who counts as a public employee for these purposes depends on historical and political definitions of which institutions count as “public” and which do not. For example, public access to salaries always extends to the personnel at state universities, despite the fact that many states now fund only a small percentage of their public university’s budget.  The obligation to disclose salaries, however, does not extend to institutions identified as “private,” no matter how much public money they receive. For example, large public-supported businesses like Lockheed Martin are nearly 80% funded through public money but have no similar requirement to provide salary and benefit data for all employees.
The landscape of which data legally contain personally-identifiable information and who cannot is clearly complicated and nuanced — and perpetually in flux. In just about all cases, the question of how to find the balance between social benefit and avoidance of individual harm remains an active area of inquiry.

Some of the leading work in finding the balance between data and harm has been developed in scientific research settings. In an upcoming post, we will explore some of the most common and useful approaches that we’ve found in this domain.

Oct 30, 2014

The FCC just began circulating among FCC commissioners an NPRM that promises to effect Sunlight's demands for more transparency in political ad spending. An NPRM is a "notice of proposed rulemaking." It's the infancy of a change in regulation, spelled out in detail and published for public feedback. Of course, some rules are bad, but this one is far from that, at least at this stage — even if it's many years late.

The initialism "NPRM" doesn't mean anything to most people, but to us, today, it means progress and success. It's the beginning of Act Two — the start of a new battle in the fight to drag money in politics out of the dark.

The whole process began years ago as the public interest sphere demanded that broadcast television stations — which receive the lion's share of political ad spending, and have the billions in revenue to prove it — should have to put their public files, including those about political advertisements, online. A simple enough suggestion that, unlike detractors' claims, did no damage to the industry and provided the public with vastly improved access to critical information about who is paying to influence votes through the public's airwaves. The FCC says that these political files have drawn millions of views (we're probably part of that).

Those television broadcaster rules went into full effect in July 2014. By the end of the month, the Sunlight Foundation — with the Campaign Legal Center, Common Cause and the Institute for Public Representation — had called on the FCC to do better, to make up for some of the time lost to hemming and hawing about the television broadcast rules and to expand these online file requirements to the other industry actors making tons of money on political ads: cable and satellite systems. The FCC responded almost immediately (in FCC terms), and on Aug. 7 called for public comment on our petition. They actually brought in radio broadcasters for good measure.

We filed our comments (in support), others filed concerns and now we're able to see that the FCC is serious about moving forward on this issue — a heartening development considering that plenty of petitions and possible rules simply starve from inaction. (Fun fact: the National Association of Broadcasters changed their tune about the whole online filing thing once they lost the TV broadcaster fight, and are now calling for the whole political ad industry to sit under the same sun. We hope to hear that same tune during the NPRM comment process.)

Needless to say, we'll continue to stay engaged with this. We'll continue building tools like Political Ad Sleuth that help people and journalists examine what billions are doing what for whom, and continue to keep the FCC on their toes about enforcing the rules that already exist.

While there are still plenty of reasons to despair about the state of political spending and money in politics, this one is a fight we are — and plan on — winning.

Oct 30, 2014

Photo credit: Yuri Yu. Samoilov/Flickr

As we audit the public data catalogs of federal agencies, we have found that some of the agencies we examined varied widely in quantity and quality of data. We also made some unusual discoveries while examining the URLs the agencies published. Here, we look at the departments of Defense and the Interior.

Department of Defense and the Bermuda Traceroute

The Department of Defense's public data catalog is much smaller than the Interior's, with only 373 datasets that offer 375 URLs for data. Of those, 59 URLs (15.7 percent) return 404 errors, including the Federal Employees Overseas Absentee Voting dataset. Additional 404 errors include links to records of FOIA requests to the Office of Secretary of Defense for 2011 and 2013. Valid URLs can be found outside of the data catalog.

But of particular interest are the domains, and, ones that appear 48 times in the catalog. The first apparently doesn't exist, or at least doesn't have a public DNS listing. The second domain has a DNS listing, but does not respond to any requests. These links may work within the department, but, if they exist at all, they block public access from anywhere else. Defense indicates that datasets exist at 24 distinct URLs among these three domains, but the existence of servers at those domains can't be confirmed. Requests to and don't result in an error from the server or a message that the server was unreachable — they provide no response whatsoever. Note that these 48 problems are in addition to, not part of, the 59 URL 404 errors above. That means, in total, 107 URLs do not function for the public — about 29 percent.

Using the traceroute tool, we can see that data packets sent to make their way across various pieces of network hardware without issue until they just disappear — a virtual Bermuda Triangle.

Perform a traceroute on and packets make their way there and back just fine. The domain is owned by an online video services company, Piksel, which makes sense given that the URLs we were examining are associated with various military video offerings, such as "The Pentagon Channel." It is unclear why this particular host would be unavailable and not respond with an error (for instance, that access is forbidden or that the page doesn't exist). It could be a side effect of compliance with Defense's directive on cybersecutity and information technology.

Again, inside the Department of Defense, that server could be accessible. But, suffice to say, this data is not publicly accessible.

Interior, Becky, Bob, Chris and Lisa

The Department of the Interior's public data catalog is impressive. It weighs in at 36,486 datasets, making it by far the largest we have examined. The catalog contains entries for the main agency and sub-agencies such as the Bureau of Land Management, the U.S. Geological Survey and the National Park Service. There were 43,940 unique URL entries associated with these datasets, and quite a few of those point to geographic databases. But when we checked those URLs, we also found a variety of issues, from items that didn't exist at the provided URL to entries that pointed to local addresses, meaning the datasets are on agency computers rather than the Internet.

For 2,650 URLs, a web server returned a "404 Not Found" response when queried with an HTTP HEAD request (this is also how we explored the Department of Defense's data). Roughly half of those came from, which responds to all HEAD requests with a "404 Not Found," meaning that the web server won't tell you if some potentially large download exists unless you attempt to download whatever it is at the URL. While that may not dissuade people from gathering a particular dataset, it is a hurdle when attempting to determine the availability of tens of thousands of potential datasets. In a manual examination of a selection of 790 URLs under the National Park Service domain, numerous URLs still "404'd" when the full resource was requested. So if you're looking to collect, for instance, information about wheelchair accessibility in national parks, you won't find information on wheelchair-accessible trails in Chaco Culture National Historical Park, N.M.. Perhaps with increased visibility, these missing resources will be located in the near future.

In addition to things that aren't there or don't appear to be, we have run across URLs that appear to be filed away on someone's desktop computer. The Department of the Interior lists URLs for files presumably on the hard drives of Bob, Lisa, Chris and Becky's computers. Here are an example of each:

In addition to Bob, Lisa, Chris and Becky's computers, there are also numerous URLs that point to shared drive letters such as G, X and I, or other locations that appear to be internal resources. Some of these URLs are part of a collection of links for a particular dataset, so there may be some useable data for those datasets, but we still don't know what's on Lisa's computer (or Bob, Chris or Becky's computer, either); it may be a duplicate of what we have access to, or, more likely, an additional piece of the dataset.

Overall, our audit reveals a pretty dismal performance by the Department of Defense, and an impressive-if-imperfect presentation by one of the biggest data publishers in the federal government, the Interior. We'll be following up on this post with additional audits in the future, so stay tuned!

Oct 30, 2014

In the final weeks of the 2014 campaign, Democrats are increasingly reliant on super PACs and their deep-pocketed donors to underwrite their effort to maintain their Senate majority. Republicans meanwhile, have money left in the party bank and a clear advantage when it comes to dark money -- funds that can't be traced to donors. These are just some of the insights we gained by visualizing data on key Senate races in a series of charts. The information comes from the Federal Election Commission via our Real-Time Federal Campaign Finance tool, which makes the up-to-the-minute campaign finance data available for easy analysis.

MORE: Confused about all the different kinds of election spending? Sunlight Editorial Director Bill Allison explains it all.

Another striking trend revealed this exercise: For all intense speculation about the fate of Senate Republican Leader Mitch McConnell in Kentucky, the unexpectedly strong threat to longtime Sen. Pat Roberts, R-Kansas, or the seemingly tightening contest for Georgia's open Senate seat, most of the big money guns in the closing weeks of the campaign have been trained on three races: North Carolina, Colorado and Iowa. Those are the three lines jumping towards the top of our first chart, which shows week-by-week spending by outside groups in competitive Senate races. Hover over each line to see the race it represents and spending details.

During the past week, spending actually dropped off slightly in Kansas, where independent Greg Orman has made national headlines with his surprising show of strength against GOP veteran Roberts, and in New Hampshire, where Sen. Jeanne Shaheen is facing a challenge from Scott Brown, the former Republican senator from Massachusetts.

In the three races attracting the most outside money, a handful of powerful outside groups are driving the bulk of the spending, with party committees leading the way. While the Republican Crossroads combine, the liberal Senate Majority PAC, environmental crusader Tom Steyer's NextGen Action Committee and the generally pro-Republican U.S. Chamber of Commerce are some of the premier forces in the races that will define these midterm elections, in the "big three" Senate races, it has been the parties leading the late charge in spending. The heavy investments in Colorado, Iowa and North Carolina from the national parties' senatorial arms are a good indication of where the smart money is moving — and other outside players are paying attention.

Since Oct. 12 — the first week we started to notice some major separation between spending on these pivotal states and the rest of the competitive Senate races — the National Republican Senatorial Committee has led the outside spending pack with $4 million investments for Cory Gardner in Colorado and Joni Ernst in North Carolina and a $5 million outlay in the Tarheel State.

The DSCC is also diverting serious resources to those battleground states but its latest report to the FEC indicates the committee is borrowing money to do so..

Looking at outside spending for the most competitive Senate races, we've sliced and diced the data to gain insights into where the money is coming from and which givers each party is leaning on hardest. For the charts below, we've looked at spending both on a cumulative basis and week by week. The latter view often highlights difference in strategy -- or fiscal exigencies. Here are four takeaways, drawn from a series of embeddable charts on campaign spending now available from our Real-Time Federal Campaign Finance tracker.:

1. Democrats caught up with GOP Senate outside spending in September, but Republicans now lead in total money spent.

The difference in spending is much starker when seen week by week (see below).

Republican groups outspent their Democratic counterparts by roughly $10 million for the weeks ending Oct. 19 and Oct. 26.

2. The difference isn't just dollars, it's who's doing the spending — and when.

Most outside spending comes from three areas: Party committees, super PACs -- which do register with the FEC and report donors -- and dark money groups, which do not report their donors. Democrats have led in terms of super PAC spending overall (see below).

The week-by-week view of super PAC spending shows a sharp drop off by Republican groups late in the campaign.

When it comes to independent expenditures made by party committees, the opposite trend is visible. During the course of this election cycle, Democratic party committees have spent more on the Senate.

But the Democratic Senatorial Campaign Committee borrowed $15 million in the final two weeks of October; as of Oct. 15 the PAC owed more than $15 million and had less than $8 million in the bank. The National Republican Senatorial Committee has spent less in independent expenditures, but by Oct. 15 had about $10 million dollars in the bank with no debt.

And while weekly spending numbers are volatile, the chart suggests that DSCC's spending seems to have slowed. That's a reversal from September and early October, when it outspent its GOP counterpart.

3. Republican groups have dominated Senate dark money spending

For much of September and October, GOP groups--typically political nonprofits--have been outspending Democratic groups by at least $5 million a week.

4. The Democrats are keeping up in outside spending thanks to one man

With their Senate committee $7 million in debt, and dark money consistently lead by their opponents, Democrats have relied on super PACs to get their message out. After months of campaigning, few super PACs have much left to spend. The one exception? NextGen Climate Action Committee the group funded almost entirely by hedge fund billionaire Tom Steyer. The three blue peaks in the chart below largely represent infusions of his cash. Steyer has given upwards of $71 million in the last two years, leaving the group with more than $16 million to spend on Oct. 15, the most of any super PAC filing a report for that period.

Oct 30, 2014

Proposals to improve public disclosure of corporate and government accountability data — from corporations' political contributions to the payments they make to foreign governments for mineral rights — attracted the most interest from members of the public, a Sunlight Foundation analysis of 10 years of public comments to two key federal financial agencies shows. Most of these comments were form letters organized by advocacy groups and unions.

The findings draw on regulatory data published by the Securities and Exchange Commission (SEC) and the Commodity Futures Trading Commission (CFTC) — data that up until now has been hard to access and analyze. That's because the SEC and CFTC are independent agencies. Unlike those of executive branch, they do not make their regulatory data available on, the source of data for Sunlight's Docket Wrench tool. Through Docket Wrench, Sunlight helps the public see what happens to laws after they pass Congress, when the government agencies write (and rewrite) the rules and regulations to implement them. Often the same organizations that are lobbying Congress on legislation turn around and do the same with agencies after the fact. The information in contained in regulatory dockets can help illuminate these patterns of influence.

With the help of a grant supported by Transparency International, and in partnership with the Thomson Reuters Foundation, Sunlight was able to integrate SEC and CFTC regulatory data into Docket Wrench. Now, the public can not only access regulatory proposals and public comments, but also see when comments form clusters of similar language, indicating they are part of a letter-writing campaign.

Like the snake that swallowed the elephant, the two financial agencies saw a huge increase in their regulatory work in the years immediately following the passage of the Dodd-Frank financial reform law, Sunlight's analysis shows. On average, the SEC and the CFTC proposed more than twice as many rules per year since the passage of the law compared to the five years leading up to it. The number of public comments they have received on these rules has also ballooned, from an average of 39,249 in the five years before passage to 185,697 afterwards, five times as much.

Docket Comments received
2011 corporate campaign disclosure docket: Comments on Rulemaking Petition: Petition to require public companies to disclose to shareholders the use of corporate resources for political activities 1,161,875
2010 oil, gas & mining payment disclosure docket: Comments on Proposed Rule: Disclosure of Payments by Resource Extraction Issuers 149,529
2013 executive pay docket: Comments on Pay Ratio Disclosure 128,297
2010 executive pay docket: Executive Compensation: Title IX Provisions of the Dodd-Frank Wall Street Reform and Consumer Protection Act 107,635
2011 executive pay docket: Comments on Proposed Rule: Prohibitions and Restrictions on Proprietary Trading and Certain Interests in, and Relationships With, Hedge Funds and Private Equity Funds 65,713
2010 oil, gas & mining payment disclosure docket: Proposed Rule: Conflict Minerals 39,928
2006 executive pay docket: Comments on Proposed Rule: Executive Compensation and Related Party Disclosure 29,659
2007 shareholder proposals docket: Comments on Proposed Rule: Shareholder Proposals 26,331
2003 security holder director nominations docket: Comments on Proposed Rule: Security Holder Director Nominations 18,257
2014 whistleblowers docket: Comments on Rulemaking Petition: Request for rulemaking petition to clarify and strengthen protections for whistleblowers, requesting the Commission hold hearings and create an Advisory Committee on Whistleblower Reporting and Protection 15,318
Top SEC dockets by comments received

However, the issue that attracted the most public comments by far — 1.2 million — was not a Dodd-Frank-related regulation but rather a citizens petition in reaction to Citizens United, the 2010 Supreme Court decision that set the stage for unlimited corporate contributions to political campaigns. In 2011, a group of law professors filed the petition asking the SEC to develop requirements that corporations disclose their political spending to shareholders and the public. “Disclosure of corporate political spending is necessary not only because shareholders are interested in receiving such information, but also because such information is necessary for corporate accountability and oversight mechanisms to work,” read the petition, which is headed by Lucian A. Bebchuk of Harvard Law School and Robert J. Jackson, Jr. of Columbia Law School.

The petition went on to say that in Citizens United v. FEC, the Supreme Court cited examples of shareholders holding their corporations accountable for its political speech. “For this mechanism to work, however, shareholders must have information about the company’s political speech.” That 2011 petition went on to garner more than 700,000 comments, and the SEC placed the issue on its 2013 regulatory agenda, the list that informs the agency's regulatory plans; however, the agency never acted on the proposal. In April 2014, Citizens for Responsibility and Ethics in Washington updated and renewed the petition. As of this publication, the docket for the two petitions had attracted 1.2 million comments.

Many of these came from letter-writing campaigns organized by groups advocating corporate disclosure of contributions. Unlike the CFTC, the SEC does its own analysis of common language in regulatory comments and posts the results on its website. The agency's analysis shows that the biggest clusters of letters were in favor of the petition. For example, this cluster of more than 380,000 letters urges the agency to issue a rule requiring publicly traded corporations to publicly disclose all their political spending. The language is identical to alerts sent out by advocacy groups in favor of disclosure, such as Common Cause and AFSCME, a labor union. Another cluster of more than 95,000 letters follows language from this alert by Public Citizen, urging the SEC to put the corporate disclosure of political contributions back on its agenda.

The issue of disclosure of payments by oil, gas, mining and other resource extractor industries to foreign governments also attracted a significant level of public attention, with 149,529 comments. (Another 39,928 comments on the subject were collected via a different docket.) The biggest cluster of letters by far, more than 149,000, grew out of by this letter-writing campaign by the advocacy group One, which works to end poverty and disease in Africa: "Please do not give into industry pressures ... and make sure that ALL companies are covered, every country and every project gets reported, and loopholes that would allow large sums of money to go unreported are closed."

The regulation in question was mandated by section 1504 of the Dodd-Frank financial reform act, and requires the SEC to issue regulations requiring the oil, gas and mining industries to publicly report how much they pay foreign governments. While anti-poverty groups hailed this requirement, industry opposed; a report from the Thomson Reuters Foundation showed that ExxonMobil and Shell executives met with SEC officials four times to discuss the issue. The two companies, along with the American Petroleum Institute and the U.S. Chamber of Commerce, later filed a lawsuit against the final regulations. Meanwhile, the European Union, Canada and Norway have already taken steps forward with requiring disclosure of such payments. In September, Oxfam filed a lawsuit in U.S. District Court to force the SEC to issue a new version of the regulations.

Another magnet for public comments: the issue of executive pay. Several dockets concerning the issue received more than 300,000 comments combined. The proposals in question also grow from Dodd-Frank provisions, including this one mandating that certain companies must disclose the ratio of how much employees are paid versus the CEO. The biggest cluster, more than 71,000, flowed in following this alert by the organization SumofUs, a consumer advocacy group whose top leadership includes former employees of the AFL-CIO. The agency has not finalized this regulation; meanwhile, there is a proposal in Congress to repeal it.

Docket Comments received
2010 position limits docket: Comments for Proposed Rule 76 FR 4752 14,179
2010 off-exchange retail foreign exchange docket: Comments for Proposed Rule 75 FR 3281 9,017
2010 position limits docket: Comments for Proposed Rule 75 FR 4143 8,395
2010 Sunshine Act meeting* docket: Comments for Sunshine Act Sunshine Act Meeting: March 25, 2010 2,850
2010 end-user exemption/swaps docket: Comments for Proposed Rule 75 FR 80747 1,525
2010 position limits docket: Comments for Proposed Rule 75 FR 67258 1,166
2006 trades reporting program docket: Comprehensive Review of the Commitments of Traders Reporting Program 1,083
2010 swap definitions docket: Comments for Proposed Rule 75 FR 80174 1,062
2010 derivatives docket: Comments for Proposed Rule 75 FR 63732 1,039
2010 swap definitions docket: Comments for Proposed Rule 75 FR 51429 964
Top CFTC dockets by comments received

Overall, the CFTC did not attract the massive letter-writing campaigns that the SEC did; this is likely largely due to the more arcane nature of futures regulation. The proposal that did receive the most comments, however, was also a Dodd-Frank mandated provision, this one on position limits, with more than 14,000 comments. The law requires limits on how many futures contracts an investor is allowed to hold in certain commodities. As Sunlight reported in 2011, a handful of groups — including some backed by petroleum marketing firms, airlines and unions — were responsible for the great majority of the comments. The biggest cluster, of more than 6,000, came from a form letter distributed by a group backed by the airline and petroleum industries. While the CFTC issued final rules, a U.S. District Court judge struck them down in October 2012 in response to a lawsuit by two powerful Wall Street trade associations, the Securities Industry and Financial Markets Association (SIFMA) and the International Swaps and Derivatives Association.

The dockets cited above should not be interpreted as a sign of intense public scrutiny of either of these two agencies. Proposals that generate large numbers of public comments are very much the exception to the rule. The great majority of proposed rules by the SEC and the CFTC attract few responses: More than half of proposed rules that received comments at the two agencies — 53 percent — received 25 comments or fewer.

Oct 30, 2014

A newspaper with the headline Open Gov National News

  • Despite predictions that the 2014 midterm elections will be the most expensive ever, it is nearly impossible to truly tell who or what is doing all that spending and who is benefiting. (NPR)
  • All of this money is making some people ask whether the massive spending is having a direct effect on polarization in our politics and in Congress. (Roll Call)
  • Tech donors are making themselves heard this election year having spent more than $20 million on donations directly to Democrats or Republicans. Chances are those numbers are a lot higher if you include independent expenditures. (National Journal)
  • Montenegro voted in favor of a new lobbying law that establishes limits on lobbying and sanctions for violations. (Montenegro)
  • The Dutch government is undertaking a data inventory that they plan to release in the spring of next year. (Open State EU)
State and Local News
  • The National Priorities Project launched State Smart, its portal to explore state level budget information. (Executive Government)
  • It can be hard to measure the cost and effectiveness of open data programs because there is significant variation across geography and levels of government. (Brookings)
  • ICYMI: A deep dive into expanding, and undisclosed, efforts by special interests to influence state Attorneys General. (New York Times)
Later This Week
  • OGIS at Five. Newseum Institute and Fri. 10/31. Newseum, Knight Conference Center, 555 Pennsylvania Ave NW, Washington, DC.
Do you want to track transparency news? You can follow the progress of relevant bills, court cases, and regulations using Scout. You can also get Today in #OpenGov sent directly to your preferred news reader. If you would like suggest an event, please email by 7 am on the Monday prior to the event.