Support all your favorite nonprofits with a single donation.

Donate safely, anonymously & monthly, in any amount. It's a smarter way to give online. Learn more
Sunlight Foundation
Washington, DC

The Sunlight Foundation uses technology and ideas to make government transparent and accountable.

The Sunlight Foundation is:

A think-tank that develops and encourages new policies inside the government to make it more open and transparent.
A campaign to engage citizens in demanding the policies that will open government and hold their elected officials accountable for being transparent.
An investigative organization that uses the data we uncover to demonstrate why we need new policies that free government data.
A grant-giving institution that provides resources to organizations using technology to further our mission and create community
An open source technology community that revolves around the Sunlight Foundation’s core mission

Sunlight Foundation is not verified as a 501(c)3 organization.

Latest News

Oct 21, 2016

EDITOR'S NOTE: We've been busy since our last digest, with most of our staff traveling out to Cleveland for TransparencyCamp last week, the presidential campaign and transition in full swing, and ongoing open data work around the United States and its cities. Sunlight is continuing to wrap up Sunlight Labs in the open, with regular updates to everyone who has reached out to us.

If you're wondering what's happening with our code or data or want to help, please email [email protected]. We'll continue to keep you all updated about the future of Sunlight as we have news to share. Thank you to everyone who has offered support, appreciation, ideas, donations and shared our ongoing work to make our politics and government more open.

In the newsletter below, we'll catch you up on what we've been up to, along with news and ideas from an election cycle in which transparency and technology has played an unprecedented role. — Alex

ONWARDS: Now that the presidential debates are over, Sunlight continues to call for more transparency and accountability from the candidates in the transition and in the campaign itself.  Our open government questions still remain open for the presidential campaigns to address. This week, we joined a coalition of other good government groups this week in pushing the Clinton and Trump campaign to adopt the ethics pledge from the 2008 Obama campaign. [Politico]


  • Voting has begun in the 2016 election. We've hit a historic milestone: For the first time, there are 200 million Americans registered to vote. While many of those records are inaccurate, the United States has made major strides in correcting the rolls. [Politico]
  • We're glad to see governors and secretaries of state across the country stand up and say that the election is not and will be "rigged." [Post and Courier]
  • A widespread DDoS attack on a DNS system this morning that led to outages in many services, including Twitter and Reddit, should catalyze some preparation to build resilience on Election Day. That should extend to newsrooms, which will be vulnerable to attack. When threat modeling includes a state actor, journalists have a enormous challenge and risks abound, as we've all seen in widely publicized hacks this fall. [Politico]
  • ProPublica launched Electionland, an unprecedented collaboration of journalists and nonprofits around the United States focused on monitoring our election in real-time. [CHECK IT OUT]
  • Lena Groeger reports that poor ballot design around the country has consequences: "We still likely lose hundreds of thousands of votes every election year due to poor ballot design and instructions," she wrote. "In 2008 and 2010 alone, almost half a million people did not have their votes counted due to mistakes filling out the ballot. Bad ballot design also contributes to long lines on election day. And the effects are not the same for all people: the disenfranchised are disproportionately poor, minority, elderly and disabled." [ProPublica]
  • Melissa Yeager reports on the spending of a super PAC, Priorities USA Action, which is working to elect Hillary Clinton. "Much of its effort has simply used Trump’s own words against him – possibly ones he said in order to gain free media coverage. The group has officially spent a whopping $100 million this election cycle, including $95 million dollars funding attacks against Trump, according to ProPublica's FEC Itemizer." [READ MORE]


  • Sunlight has long called for Congress Research Service reports to be published online. Now, a new website from Demand Progress has done what Congress should have done itself many years ago, making thousands of CRS reports freely available to anyone. [READ MORE]
  • The open data maturity model Philip Ashlock described to Samantha Ehlinger is a sensible next step for and open government in the USA. [Fedscoop]
  • Speaking at the last week's New Frontiers Conference, President Obama weighed in whether government can (or should be) be run like a startup or tech company:
    The final thing I’ll say is that government will never run the way Silicon Valley runs because, by definition, democracy is messy.  This is a big, diverse country with a lot of interests and a lot of disparate points of view.  And part of government’s job, by the way, is dealing with problems that nobody else wants to deal with.

    So sometimes I talk to CEOs, they come in and they start telling me about leadership, and here’s how we do things.  And I say, well, if all I was doing was making a widget or producing an app, and I didn’t have to worry about whether poor people could afford the widget, or I didn’t have to worry about whether the app had some unintended consequences — setting aside my Syria and Yemen portfolio — then I think those suggestions are terrific. That's not, by the way, to say that there aren't huge efficiencies and improvements that have to be made.

    But the reason I say this is sometimes we get, I think, in the scientific community, the tech community, the entrepreneurial community, the sense of we just have to blow up the system, or create this parallel society and culture because government is inherently wrecked.  No, it's not inherently wrecked; it's just government has to care for, for example, veterans who come home. That's not on your balance sheet, that's on our collective balance sheet, because we have a sacred duty to take care of those veterans.  And that's hard and it's messy, and we're building up legacy systems that we can't just blow up.
  • After directly engaging with the agency and seeing our analysis amplified by media outlets, we're glad to share that the Department of Veterans Affairs acknowledged the issues we raised in September and has committed to addressing our concerns by producing a new open government plan in December and restore hospital performance data. We'll be watching — and trying to get Interior, DHS and Treasury to honor the spirit and substance of President Obama's Open Government Directive. [READ MORE]
  • Former White House official Nick Sinai, who inherited the open data portfolio in the Office of Science and Technology Policy from Beth Noveck, wrote about a recent panel he hosted at the Harvard Kennedy School on the Freedom of Information Act (FOIA) in a digital age. He connects a lot of dots regarding the Obama administration's efforts but leaves out crucial context regarding FOIA reform passing Congress: the Department of Justice lobbied against it in 2014, delaying passage until this year, and did not follow through on the U.S. National Action Plan commitment to build a new FOIA request portal in 2015 -- and the White House never supported any FOIA reform legislation publicly until bills passed both the House and Senate this summer. Now that Congress has instructed the Office of Management and Budget to create one, we hope that 18F's work on the pilot can be revived and share Sinai's hope that agile development will lead to a platform built with the requester and FOIA officer communities, nor for them. [Medium]
  • Here's a priority for the next White House: Systematically map and publish data about factory farms through the United States so that the Environmental Protection Agency, Justice Department, U.S. Department of Agriculture and the Food and Drug Administration can more effectively monitor, oversee and regulate them. [Inside Climate News]


  • How much gun violence occurs in your Congressional district? The Gun Violence Archive (GVA) has a new “Congressional Reports” tool that can give you some answers. Sunlight's Emily Shaw: "By collecting information from more than 1,500 sources, GVA has created a high-quality alternative to the public sources of information about gun deaths and injuries that Congress has limited. GVA’s database provides near real-time incident-level data on firearm deaths and injuries, including information on characteristics of the incident participants (age range, sex) and the incident itself (general location, context of firearm use). It also provides the informational source of the database entry, so users can gather even more context about individual cases if they are interested." [READ MORE]
  • Delaware launched a new open data portal. [StateScoop]
  • If you want to bring TransparencyCamp to your city, Sunlight has an answer: adopt and adapt our approach to convening, hosting and running open government unconferences. Greg Jordan-Detmore: "We have a detailed, publicly available set of instructions on how to run your own TransparencyCamp, thanks to the hard work of Laurenellen McCann and other former Sunlighters. We’re happy to announce that we’ve now added these to GitHub. Just like our tech tools, it’s open source — go ahead and fork it!" [READ MORE]


  • The impacts of hacks and leaks needs to be thought through very carefully, as Jonathan Zittrain explains. [Just Security]
  • Radical transparency that sacrifices privacy in the pursuit of political sabotage damages open government globally. [WSJ]
  • "Responsible data" is a healthy frame for this discussion. It puts people in mind of something real, not abstract: being entrusted with something. Whether you're a steward, protector, owner, consumer, publisher, reporter or developer, you should know how to be responsible with data, particularly data that has been lost, leaked, or stolen. I'm honored to be quoted in this piece, on transparency and ethics. This set of issues is important and relevant to far more than the 2016 US election. [Engine Room]
  • Who else will shape the future of a "data society?" Jonathan Gray digs into the issue and points to a new effort: "As a modest contribution to advancing research and practice around these issues, a new initiative called the Public Data Lab is forming to convene researchers, institutions and civil society groups with an interest in the making of data infrastructures, as well as the development of capacities that are required for more people to not only take part in the data society, but also to more meaningfully participate in shaping its future. [OKFN]
  • Following up on the International Open Data Conference, here's an argument to make open data more evidence-based. [IODC]
  • The Economist Intelligence Unit has a new report out on "empowering cities," looking at the role people play in the "smart city" context. [White Paper] [READ MORE]
  • In other weekend reading, here's another new white paper on "reframing data transparency." If you dig in, let us know what you think of their frame. [Hunton and Williams]
  • The United Kingdom is putting a renewed focus on digital inclusion, including a dashboard to track it. [Digileaders]
  • The state of open data in Germany appears to be improving. [DataEconomy]


Tired of your boss/friend/intern/uncle forwarding you this email every morning? You can sign up here and have it delivered direct to your inbox!

We want to find and share the most important stories about open government around the world from the past 24 hours here. To do that, we'll need YOUR help. Please send your tips and feedback at [email protected]. If you would like to suggest an event, email us by 7 a.m. on the Monday prior to the event.

Oct 20, 2016

A logo for Priorities USA Action in front of an outline of America.
(Image credit: Priorities USA Action)

There’s little disagreement that this has been a crazy and unpredictable election cycle. Although money in politics was a focal point early in the campaign, the lack of spending by Republican nominee Donald Trump during the primaries had some questioning whether outside spending still mattered given the failure of Right to Rise, the colossal pro-Jeb Bush super PAC, to counter such a polarizing opponent. (We’ll be watching for any more developments over the next few days as the last round of pre-election campaign finance reports are filed with the FEC.)

Right to Rise spent $86 million dollars in independent expenditures supporting the failed candidacy of Bush. For the most part, the super PAC took the high road; $83 million of that total went toward efforts boasting of Bush’s record and qualifications to be president.

Meanwhile, Trump was getting earned media through the outrageous things he said, calling into 24-hour cable stations and making appearances on shows like Saturday Night Live. As we pointed out, it was a pretty ingenious approach to the primary. But hindsight is 20/20, and perhaps that strategy was strengthened by Republicans who didn’t want to take a strong swing (at least through super PACs) at a fellow Republican nominee.

Now in the general cycle we’re getting an idea of what that strategy looks like when opposed by major super PAC spending.

For the first time, Trump is actually getting hit by a strong opposition super PAC campaign. Priorities USA Action, which backed President Barack Obama in 2012 and includes such liberal megadonors as George Soros, Fred Eychaner and Donald Sussman, is working to elect Hillary Clinton. Much of its effort has simply used Trump’s own words against him – possibly ones he said in order to gain free media coverage. The group has officially spent a whopping $100 million this election cycle, including $95 million dollars funding attacks against Trump, according to ProPublica's FEC Itemizer. And look for this number to increase in the weeks leading up to Election Day — Priorities USA Chief Strategist Guy Cecil stated that more money is being poured into crucial swing states starting today.

On the other hand, the Trump campaign did little fundraising during the primary and had to scramble to load its coffers for the general election. Few outside organizations supported him initially, so it isn’t surprising that outside groups have spent $178 million opposing him and only $84 million supporting him, according to Meanwhile, Clinton has had nearly $50 million in outside spending against her thus far compared with $37 million supporting her.

But let’s put that in context by comparing to 2012. Mitt Romney had an equal amount, $90 million, spent both opposing and supporting him. On the other hand, Obama was subject to a whopping $333 million opposing him and only had $49 million in his favor — but won the election in spite of all this.

It’s hard to say what all of this means when you have a candidate who isn’t afraid to get his hands dirty himself. In a typical election, candidates let their super PACs attack their opponent so they can distance themselves from the attack as well as not have their own words used against them in attack ads. Trump, breaking with campaign tradition, clearly had no fear of either of these things, and even seems to relish in incorporating attacks normally reserved for super PACs and dark money groups in his stump speeches.

Will the super PAC money really matter, or will the voters brush off this media blitz? The ultimate referendum, of course, happens on Nov. 8.

Oct 19, 2016

With, this taxpayer-funded research is now available to the public in an accessible way.

It has been a long held position by Sunlight and our allies that the Congressional Research Service (CRS), a taxpayer-funded research arm for Congress, should make its reports to the available to the public. But until Congress expands access to this public knowledge, a newly launched resource from Demand Progress,, will enable the disclosure that our legislators should have enacted years ago. Sunlight's founding executive director, Ellen Miller, wrote in 2009 that opening up CRS reports "is an easy transparency reform that boggles the mind as to why it has not yet been done."

The research that CRS conducts and provides to Congress is a valuable resource that, if published online, would empower the public to be more informed about the subjects and the counsel that our representatives receive. We are far from alone in this view.

“It is 2016, any student, reporter, taxpayer or interested citizen should be able to view CRS reports online," said Rep. Leonard Lance, R-N.J., in a statement. "These reports for paid for by taxpayer funds, the taxpayers should be able to read them. It is past time to end the era of secrecy to these reports and open them to the benefit of research, reporting and public information. This online portal will become a resource for many and bolster the argument for transparency.” was created by Demand Progress and is the brainchild of its policy director, Daniel Schuman, a former Sunlighter whose tenure here featured years of advocacy to open up legislative data and congressionally mandated research to the public. Schuman, who once worked as a legislative attorney at the CRS on Capitol Hill, came to believe that CRS reports should be available and has now acted to make that a reality when Congress did not.

“For more than 20 years, the public has clamored for Congress to systematically release CRS reports to the public," said Schuman in a statement. "Instead, those with DC connections have received preferential access, leaving lone members of Congress to fill the gaps and address iniquities. Congress must do better, and this new website points the way forward.”

This isn't the first time that advocates have built what Congress did not. From 2005 to 2013, the Center for Democracy & Technology provided provided public access to CRS reports that in the public domain at The site was taken offline in 2014 due to security problems. In its absence, the Federation of American Scientists has been maintaining the best online archive of CRS reports, but it isn't complete and there are some issues with usability. includes over 8,200 reports in its searchable database, with automatic updates, RSS, "freshness ratings" that show how much a report has changed, a bulk download function and an open source platform that the public can adopt and run. The site's administrators are mindful to the need to protect privacy while releasing open data: They have redacted the author's name, email and phone in these reports. They've also made it clear that the reports are not copyrighted.

"Congress long ago could have fixed this inequitable and anachronistic situation with ease and little to no cost," said Kevin Kosar, senior fellow and governance project director at the R Street Institute and a former research manager and analyst at the Congressional Research Service, in a statement. "But it dithered for years, so the private sector stepped forward and got the job done in a blink."

If you're wondering how the CRS reports for the site are being acquired, the process relies on a partnership with a Democratic and Republican member of Congress, each of whom are providing the reports to Demand Progress. This "disclosure hack" is making use of a legal disclosure mechanism: every member of the public can request a CRS report from his or her Representative.

It is refreshing to see some members of Congress committed to a more open government in this area.

“Increasing transparency and accountability in government is not only the key to improving public trust, it is the key to improving the way government works," said Rep. Mike Quigley, D-Ill., in a statement. "I applaud Demand Progress on their efforts to bring greater transparency to the exceptional, non-partisan research conducted by the Congressional Research Service. Information is the power the people need to trust their government and the work we do each and every day.”

We have heard the arguments that publishing CRS reports online would have a negative impact upon the speed or quality of the information that Members of Congress and are well aware of the opposition that their release received on the floor of the House earlier last year, when legislators voted against open CRS reports to the public.

As Matt Fuller reported for the Huffington Post, "At a time when highly informed voters might seem like a good thing, the Appropriations Committee voted down, 18-32, an amendment from Reps. Mike Quigely (D-Ill.) and Scott Rigell (R-Va.) that would have made it easier for the public to access Congressional Research Service reports."

We and our allies judge the arguments opposing expanded access to CRS reports to be specious. Sunlight stands with a bipartisan coalition in calling on the U.S. House of Representatives and the U.S. Senate to work with the new Librarian of Congress to make CRS reports open to all. (Perhaps at

“Today, Demand Progress has enabled researchers and entrepreneurs everywhere to benefit from the kind of information that drives innovation," said Julie Todaro, president of the American Library Association, in a statement. "Now it’s time for Congress to permanently guarantee this access by law, and to itself create an online portal to such vital, taxpayer-funded information for all.”

Oct 19, 2016

The exterior of a VA medical center.
A VA medical center in Salt Lake City, Utah. (Photo credit: Veterans Health/Flickr)

At the end of September, Sunlight reported that the Department of Veterans Affairs (VA) needed a new open government plan after failing to produce one since 2010, and had stopped publishing performance data.

After directly engaging with the agency and seeing our analysis amplified by media outlets, we're glad to share today that the VA has acknowledged the issues we raised and has committed to addressing our concerns.

"Though VA has widely embraced the guidelines’ spirit of transparency, it has not updated the Plan every two years as required," said Victoria Glynn, deputy press secretary at the VA. "The VA plans to remedy this immediately by producing a new self-assessment, progress report, and next steps by mid-December 2016."

We look forward to reading and evaluating all three of these components, which the U.S. chief information officer and U.S. chief technology officer required in the White House guidance issued in July. We're glad to see that the VA will join other Cabinet agencies in complying with President Obama's Open Government Directive and hope that their self-assessment and progress report will be accurate, honest and reflective of the department's record over the past six years, along with the concrete steps it will take to make the services and staff that support veterans and their caregivers more open and accountable to the public.

That plan and the agency's commitments will include disclosing data about how well VA services are working, where and when. On that count, Sunlight called for the VA's Hospital Compare website to go back online and the data on it to be restored to the Center of Medicare and Medicare Services (CMS) Hospital Compare website. If the public searches for any VA hospital, we can all see missing performance data.

In its statement to Sunlight, the VA disputed our characterization of the hospital data as missing and highlighted ongoing data releases. While the agency acknowledged ongoing issues regarding publishing VA data to the CMS website, it highlighted that they have continued to post the results of inpatient and outpatient quality measures at

"As a truly accountable and transparent organization, VA generates and releases a great deal of data," said Glynn. "For example, every two weeks, we release a large set of health care data related to patients’ access to care, including wait times. The most recent was October 13. On the healthcare data you were looking for: to put the bottom line up front, we have been releasing all of the data you’re looking for, just in a different place than perhaps you expected."

The VA told us that after it finishes "working through legal and technical details," however, it will once again begin publishing data about hospital quality performance to the HHS starting on Oct. 19.

The agency also highlighted other datasets, including comparisons between VA and the private sector for outpatient care and information about how individual VA facilities perform, and welcomed public feedback on these datasets.

"We are committed to full transparency about all dimensions of VA performance, and believe Americans should know how VA health care performs compared with private sector health care organizations," said Glynn. "Our ultimate goal is that all Veterans receive timely, safe, high-quality care wherever and whenever they seek our assistance."

Acknowledging problems with transparency and disclosure is crucial to start making improvements. We commend the VA for responding to our analysis with substantive feedback and look forward to accountability for following through on these commitments.

Oct 15, 2016

Bill Hunt present at TCamp16.
Sunlight's Bill Hunt encourages attendees to host their own TransparencyCamp.

One of Sunlight’s most loved and well-known initiatives is TransparencyCamp, our annual unconference focusing on open government. What if it was held more often?

There are so many people and organizations who are passionate about government transparency and open data, some who have attended past TransparencyCamps and some who have yet to experience its awesomeness. Let's empower them.

We have a detailed, publicly available set of instructions on how to run your own TransparencyCamp, thanks to the hard work of Laurenellen McCann and other former Sunlighters.

We’re happy to announce that we’ve now added these to GitHub. Just like our tech tools, it’s open source — go ahead and fork it!

Here at Sunlight, we are big believers in driving change from the ground up. The unconference format itself already represents a participant-driven model. The open source nature of TransparencyCamp is taking it a step further. You can run the TransparencyCamp you want to see. (And yes, you can use the “TransparencyCamp” name.)

TransparencyCamp started in 2009, drawing inspiration from other unconferences like Foo Camp and BarCamp. The ecosystem of unconference “camps” has expanded since then. In 2011 — inspired by TransparencyCamp and CityCampOpenPlans started TransportationCamp, an unconference focusing on the intersection of urban transportation and technology.

My colleague Stephen and I traveled to New York for TransportationCamp in late September, and I attended one in Massachusetts in April. What amazes us about TransportationCamp is the frequency of events. There have been 10 TransportationCamps so far this year, with another planned for November.

TransportationCamp started in 2011 with a pair of events in New York City and San Francisco, thanks in part to support from the Rockefeller Foundation. Another was held in Montreal later that year, followed by one in Washington, D.C., the next year. Since then, TransportationCamp — whose coordinator has changed from OpenPlans to Mobility Lab — has rapidly expanded, with three events in 2013, four in 2014, six in 2015 and 11 in 2016.

The large number of TransportationCamps is thanks to its structure: Each event is proposed and managed by its own local organizers, who deal with staffing, fundraising and logistics. Mobility Lab’s role is largely limited to electronic communications, like the website and Twitter account.

We love how well TransportationCamp has spread organically around the country and the world thanks to local groups taking the charge on organizing their own — and we’d love to see the same happen for TransparencyCamp. We’re already off to a great start this year. In June, Open State Foundation and the Presidency of the Council of the European Union ran TransparencyCamp Europe in Amsterdam. (The event wasn't officially affiliated with Sunlight.)

While most of our past TransparencyCamps have been here in Washington, D.C., we just wrapped up a successful TCamp16 in Cleveland. We hope to see more TransparencyCamps in cities all over, just like TransportationCamp we can’t wait to see what you come up with.

Oct 12, 2016

Tables full of guns while interested consumers peruse the options.
(Photo credit: M&R Glasgow/Flickr)

There are few issues more important to Americans than gun policy. According to Gallup, 72 percent of American voters identified gun policy as being “very important” to their 2016 vote decision. Recent polls also show more than 80 percent of Americans see gun violence as a serious or very serious problem.

Despite the high levels of interest in this issue, Congress decided to bend to the NRA’s preferences and has severely limited the federal government’s ability to conduct research on gun violence over the past 20 years by banning any gun-related research that could be policy-relevant. As a result, the state of data collection on gun violence is abysmal. Without good and timely information, it’s hard to press for specific changes.

In absence of reliable, detailed information about gun violence, projects like the Gun Violence Archive (GVA) have stepped into the breach. By collecting information from more than 1,500 sources, GVA has created a high-quality alternative to the public sources of information about gun deaths and injuries that Congress has limited. GVA’s database provides near real-time incident-level data on firearm deaths and injuries, including information on characteristics of the incident participants (age range, sex) and the incident itself (general location, context of firearm use). It also provides the informational source of the database entry, so users can gather even more context about individual cases if they are interested.

While GVA is not an advocacy group, it provides information that is unquestionably useful in the current political moment. We were pleased to see that they were able to provide their national data in chunks that are highly relevant to political action. GVA has announced that they are now making GVA data on gun violence incidents available for download by congressional district.

Choosing the “Congressional Reports” tab at the top of the screen allows you to choose your state and then congressional district of interest. This selection then returns information about the members of Congress serving that district as well as a “stats” table which lists the total number of firearms incidents and mortality, the number of children and teens injured or killed by firearms, and certain contexts of interests (including firearms use in mass shootings and self-defense) that have occurred since 2014. Clicking on the state’s “Stats” table, which on its surface provides just aggregate data, brings up a page with incident-level data for the congressional district. The incident data can be explored interactively. Each case links to more detailed variables and related news sources. The basic incident-level data can also be downloaded as a .csv file.

If you are not interested in congressional districts, or you’d like to compare a single district to the state in general, the "Congressional Reports" screen also facilitates your access to datasets for an entire state. By selecting a state and then choosing “all districts,” you can also get the state’s entire gun-related death and injury data for 2014-2016.

Because GVA includes not just deaths and injuries, but also threatening events in which shots were fired but nobody was injured, this data provides a more comprehensive picture of gun use than a data source which solely provides deaths and injuries. It is clearly information that anyone could use to evaluate the state of gun violence in their own location, or across different locations with different gun laws.

Here at the Sunlight Foundation, we really love open data and we have a particular interest in open government data. It’s data that we know we should have a basic right to, since we pay for it through our taxes and because it’s on a subject of great public interest.

When political leaders refuse to produce and and publish the public data we want, we’re grateful that private groups can step in to help out. For those of us who want data on the issue of gun violence — in our districts, in our states, and across the country — the GVA’s new Congressional Reports tool allows us to get the information we really need in order to thoughtfully analyze the issue.

Oct 12, 2016

For the past few weeks, we’ve been hard at work wrapping up Sunlight Labs. As the senior technologist on staff, I’m focused on two priorities: getting our projects into a state where they can easily be used and adopted by others, and moving our core websites into solutions that don’t require a development team to maintain.

Project wrap-up

Our primary focus has been on our core, currently active projects: Open States, Email Congress, Political Ad Sleuth, Political Party Time and so on. There are dozens of smaller tools that comprise these larger projects, as well as many, many other applications we’ve written over the years to solve various problems. In total, we have several hundred individual projects. About half of these projects lived in a git repository on our own internal GitLab server, and the rest were already on GitHub. Most everything is currently running on AWS or Heroku.

To make these projects ready for re-use, we have to do several things:

  1. Move all of our projects to GitHub.
  2. Add licenses to the projects. Without a license, our code isn’t open source and can’t be reused. Following Sunlight’s ethos, we’ve chosen GPL 3.0 as our default license.
  3. Pull out any secure credentials, API keys, passwords and other private information that shouldn’t be shared with the world.
  4. Document our projects so that everyone can tell what they’re for. Lots of our projects have clever, but not necessarily helpful, names that do not give any indication of what they do.
  5. Export all of the publicly shareable data from the live running projects, and put this somewhere that people can use it.

I have created a collection of tools to help automate many of these steps whenever possible. I was able to move the many projects to GitHub without too much trouble thanks to the GitHub and GitLab APIs. From there, I created a quick script to take an inventory of what we did and didn’t have for documentation and licensing. Not all of the repositories are public yet, as we need to finish steps two and three.

Adding licenses was a bit more tricky, as some of our projects were primarily static assets and not suitable for GPLv3 — in which case we’d prefer Creative Commons (either CC0/Public Domain, or Attribution and Share Alike). For that, I needed an interactive tool for creating licenses wherever they were missing, based on user input.

Scraping out secure credentials was a similar problem, but complicated by the fact that we needed to remove these from the entire history of the git repo. As a first step, I looked for obvious configuration files and flagged ones that contained secrets. Then there was quite a bit of manual double-checking as API keys had frequently been left in random source files. After creating example configuration files and removing credentials, I used the BFG tool to cull these from the entire history of the projects.

With that being mostly done, we’re now reaching out to our many Labs alumni to help us create documentation for our projects that need it. Since everything is now on GitHub, it’s as simple as filling a pull request! Please feel free to contribute if you have something you can add to this effort.

We’ll be holding off on exporting the data until after TransparencyCamp next week so that we won’t have to do this repeatedly as new data comes in. We hope to put everything we can on GitHub or in collections within the Internet Archive.

Site migration

We also have a number of other content-driven sites, including the Sunlight Foundation main website, the TransparencyCamp site, Open Data Policies Decoded and others. Almost all of our sites and projects are custom in-house projects, built with Django, Flask or other tools. Currently, these all require a programmer to make changes to the sites and require a complex setup on AWS.

Wherever possible, we’re attempting to transfer these to platforms that staff can manage  without technologists. In many cases we’ll be switching to Wordpress or GitHub Pages for the hosting of our content-driven sites. There are certainly other great tools out there, but we’re deliberately choosing two of the most popular ones, so help will be easy to find.

Looking forward

We have a lot to accomplish in the weeks remaining before we wind down Labs. Having watched so many projects shut down recently, my goal here is to preserve as much of the legacy of Sunlight Labs as possible, for future work to build off of. At The OpenGov Foundation, I brought an ethic of open by default to all of our work, and it’s in this same spirit that I’m approaching the closing of Labs — using solid, open source principles to make sure these tools are available to the community for years to come.

No one can predict when a project or organization will end when they’re just starting out. Making sure that you have a plan for your work, with a logical beginning, middle and end, is critical. The tech world is increasingly unpredictable – make the effort now to make sure your work is preserved for the future. Whether you’re a nonprofit, a for-profit or a government agency, open whatever you can, however you can!

Oct 11, 2016

EDITOR'S NOTE: Thank you for the kind notes, inquiries and calls over the past week while Today in OpenGov was offline as your faithful correspondent traveled to Spain for the International Open Data Conference. (Look for our dispatch from Madrid below.) Sunlight's staff have been busy in the interim, preparing for TransparencyCamp this weekend in Cleveland, Ohio (you can still register and submit a session idea), developing municipal open data policies with the public and cities, and working with our allies and the public around the world to hold governments accountable. Please keep your tips, ideas and feedback coming as we get back to publishing daily. -- Alex

LABS UPDATE: As you may have heard, GitHub and the Internet Archive stepped up to help preserve all of the code and data created by Sunlight Labs over the past decade. Labs Director Kat Duffy has been in touch with many of you over the past several weeks. As she notes on our blog, Sunlight is still exploring new homes for existing projects: "We’ve received dozens of offers of assistance and interest in projects like Open States, Hall of Justice, Politwoops and Email Congress. I’ve responded to those who have reached out through [email protected] and will be keeping interested parties apprised of our progress. Our board will make the final determination on such decisions, and I will continue to work with our Interim Director John Wonderlich and the board to facilitate that process." Please keep in touch with her as we work through this. [READ MORE]


  • As most of the connected world knows, the past week has been one of the most tumultuous periods in modern presidential politics, with two presidential debates and publication of video and email online shifting the course and context for the election. Like you, Sunlight is watching the debates and following the impact of these disclosures and their manner. We've been calling for all  candidates to disclose their tax returns, in keeping with the transparency norms established over the past 4 decades. Last week, the New York Times published 3 pages of Republican presidential nominee Donald Trump's 2005 tax returns, which answered some questions of public interest. Many remain unanswered. [READ MORE]
  • We're also asking for more transparency and accountability from the candidates and in the debates themselves: We'd like to hear answers to open government questions from all of the presidential campaigns. [READ MORE]


  • Former Speaker of the House John Boehner has been cashing in on his connections and influence in a new role as a "strategic advisor" to tobacco giant Reynolds American due to lobbying loopholes. [READ MORE]



  • It’s been much more than a century in internet time since the founder of the World Wide Web laid out a vision for countries to unlock their files and share them with the world. Here's 10 of our takeaways from the recently concluded International Open Data Conference about where the efforts stands. [READ MORE]
  • Billions of people know that the oil industry generates enormous profits, and therefore the opportunity for enormous corruption. Sunlight's Joy drilled deep into how open data can bend the curve of corruption, adding much-needed transparency and accountability to an energy sector that needs it. [READ MORE]



Tired of your boss/friend/intern/uncle forwarding you this email every morning? You can sign up here and have it delivered direct to your inbox!

We want to find and share the most important stories about open government around the world from the past 24 hours here. To do that, we'll need YOUR help. Please send your tips and feedback at [email protected]. If you would like to suggest an event, email us by 7 a.m. on the Monday prior to the event.