Dark money has haunted the psyche of good government reformers. Recent changes in campaign law raise the prospect of unlimited donations, routed to political action committees through 501(c)4 "social welfare" organizations that don't have to disclose contributors' names. That could allow innocuously named groups to shelter powerful individuals and corporations and the influence they're exercising to determine who wins a federal office.

So far, that scenario has been the dog that didn't bark, but that doesn't mean it has been defanged.

Sunlight looked at the super PAC filings in search of 501(c)4 dark money and found just one outright donation listed for 2011. More common were in-kind contributions in the form of staff and office space shared by 501(c)4s with sister organizations. Total receipts reported by super PACs from 501(c)4s amounted to $1.6 million, a number dwarfed by the overall spending figures. Still, it's not insignificant -- and it means super PACs have that much more money to spend on ads because their operating costs have been covered by other organizations.

"Super PACs are still a relatively new phenomena, they are still evolving. Under the current rules, there are a multitude of ways to make anonymous contributions," says Mimi Marziani, counsel for the Brennan Center for Justice. Other routes for dark money are emerging:

• Some PACs have been getting donations from companies that only exist on paper or, as the New York Times reported, are very hard to find information about .
• And, as Dave Vance of Campaign Legal Center points out, social welfare organizations "can spend tens of millions dollars of dollars on TV ads anyway, so they don't need to give to another source." One example of a 501(c)4 entering a political race without having to report donors: Citzens for a Working America Inc., reported spending $475,000 on ads supporting Mitt Romney in Iowa, but listed no donors to the FEC at the end of the year. That's because 501(c)4s do not have to disclose the names of political donors names unless they explicitly earmarks his or her money for a particular political advertisement. The FEC last year was asked to change this regulation to force broader disclosure of donor names, but declined to do so.   
• A study this week by the Washington Post using television advertising records provided by Kantar Media-CMAG suggests that more than a third of political advertising aired in the presidential campaign so far this cycle as been underwritten by social welfare groups who will never have to disclose the sources of their funding. 

Read all about it

Sunlight Executive Director Ellen Miller responds to this guest post by Mike Godwin:

There is little in Mike Godwin's response that we disagree with. As he writes, the debate over SOPA and PIPA was changed not by "politics as usual" or a late infusion of interest group lobbying cash, "but the participation of the online community, including Wikipedia, Reddit, and others, to let policymakers know about their unhappiness with the direction and process of the legislation." Amen to that. Our blogger, Lee Drutman, did not argue that this was not grassroots or that it was solely organized by Google or other tech lobbies. What he did write was, "Facebook, Google, Wikipedia and tens of thousands of others changed the scope of conflict in a flash. Harnessing the unique megaphone they had built as content providers, the online companies generated as many as 3 million e-mails to Congress, 7 million signatures and 3.9 million tweets. It was enough to cause many in Congress to wonder whether they had unwittingly kicked a hive. [Emphasis added]

And Drutman took care to note that something indeed had changed in the wake of the SOPA/PIPA fight: internet content providers on their own had managed to mobilize public attention. Drutman wrote:

It used to be the case that for this to happen, the mainstream media had to be involved in calling attention to the travesty. Those days are disappearing, and this is generally a good thing. By lowering the barriers to entry, the Internet has the potential to make political activism more democratic than ever before, and the SOPA turnaround is proof that it can work politically to engage a broad crowd in short order.

Where perhaps there is some disagreement between us and Godwin, whose work with EFF and Wikimedia we greatly respect, is on which sy-LA-byll to put the em-PHA-sis. Godwin argues that the campaign contributions and lobbying spending by tech companies was a minor or unimportant factor in the contours of the SOPA/PIPA fight; Drutman wrote that it could not be ignored and that the reality of the dynamics of power in Washington hadn't changed quite as much as some would hope. Indeed, consider this counterfactual: last year, when Google cut its deal with Verizon and the FCC over "net neutrality," there was also a howl of online protest, but that time nothing changed. The big money players were aligned instead of being divided, and grassroots opposition alone was not enough to prevent the deal from happening.

That said, we agree with Godwin that something else is also going on here that ought to cheer all fans of small-d democracy, of which we count ourselves. Because of the open, two-way nature of the Internet, as well as its large non-commercial sector, the SOPA/PIPA fight may mark the emergence of the networked public sphere as its own interest group. While some big tech companies joined the fight against the bills on their own, internet users rallied themselves too, and then pushed the bigger companies to join in, or risk losing their users' allegiance and business. In that case the story is not the traditional clash of narrow private interests, but the rise of a networked public sphere that has its own "lobby" that isn't just a business interest but more of a real public interest.

A Sunlight analysis of the fight on Capitol Hill over SOPA is generating some pushback in the online community from activists who think we overstated the role of money and corporate lobbying in the debate. In the interest of broadening and deepening the conversation, we asked one of our critics, Mike Godwin, a former counsel for the Electronic Frontier Foundation and Wikimedia, for permission to print his counterpoint:

I believe that Sunlight (and one of its primary sources, OpenSecrets.org) missed the story. Just as I would not write an Occupy movement story grounded in how much money was spent for food, medical care, and tents, I wouldn't write about a "net-roots" popular movement focusing on the convenient fact that money was spent inside the Beltway during the time that the popular movement seems, temporarily, to have given some tech companies some traction on one issue.

It's well-established that Google's estimable DC presence -- their many dollars and their top-notch personnel -- had little effect on the ETAs of the SOPA and PIPA legislation before the holiday break. What changed the debate was not "politics as usual" or an infusion of cash, but the participation of the online community, including Wikipedia, Reddit, and others, to let policymakers know about their unhappiness with the direction and process of the legislation. This response was not organized by Google or any tech money at all (except perhaps the meager salaries that tech-policy writers tend to receive).

Response: Sunlight Executive Director Ellen Miller replies.

One thing I did notice as a close observer of the blackout and abrupt opposition to SOPA and PIPA was how quickly the Motion Picture Association of America  and other content-company spokespersons adopted and propagated the narrative that the net-roots opposition was of Google's making. "Google owns the platform," I heard more than once from people who may be unclear on what "own" and "platform" might mean in the Internet context. It disappointments me greatly to see how smoothly Sunlight adopted and promoted a version of the narrative that this is just one set of industrial interests competing with another.

I spent more than a decade in Washington dealing both with content-company lobbies and with tech company lobbies, and a proper comparison needs allocation breakdowns, not rough totals (and not even broad totals over time). The range and substance of issues on which MPAA maintains a lobbying presence are not the same as the ones that tech companies invest in. They are not even the same as the ones that Google invests in.

To give you a comparison: the pharmaceutical lobby, the American Medical Association, the American Library Association, and the agricultural lobby may speak from time to time on the same issue, but we would be cautious before saying that the raw numbers they spend tell us anything about about comparative influence, or whether the outcome of a policy dispute is a function of investment of dollars (or, as we are perhaps too fond of saying, "politics as usual").

Now, nothing here should be taken as opposing discussion or criticisms of how money is spent in Washington -- quite the contrary. I believe citizens are actually capable of more finely tuned, granular analysis of where the money and influence is applied. I do not see that granularity or skepticism in Sunlight's article. I want breakdowns, names of lobbyists, how many visits took place, who visited whom, what was discussed, and so on. (From my own DC days, I know that most if not all of this is documentable.)

Now, once the the blackouts and other responses were being prepared and were implemented, certainly this enabled well-moneyed and well-established  interests to say to policymakers, "Hey, this non-controversial stuff you've fast-tracked may not be so uncontroversial after all."  And the fast and furious backtracking by supporters underscored what the policymakers were hearing. So you should not read me as saying that Google et al. simply got out of the way of the disgruntled public. There was synergy there, as there generally is in multifactorial human political events. The public protest enabled tech companies to say that the issue was a real one for individuals.

But the reductive treatment of the money issue by Sunlight obscures this. The blackouts would have occurred regardless of what checks the tech companies were writing and in what amounts. It is hugely important, because we believe (as we suppose Sunlight does) in the potential for democratic activism, and that it does us no service to reduce this policy disagreement to a question of who employed more lawyers. (Besides, as a lawyer who practiced both on Capitol Hill and at the Federal Communications Commission before moving West, I can assure you I know who sends these places more lawyer-lobbyists per week on media-related or copyright-related matters, and it's not Google or Microsoft or Apple or Intel or Cisco.)

And I can assure that the final outcome of this fight has yet to be determined. We may reasonably expect versions of SOPA and PIPA to return, only more felicitously named, and less transparently, once we're past this political year.

Policy Fellow Matt Rumsey wrote this post.

Here is Tuesday's take on transparency-related news items, congressional committee hearings, transparency-related bills introduced in Congress, and transparency-related events.

News Roundup:

Relevant committee hearings scheduled for 2/7:

House:

• Budget Hearing- Library of Congress, U.S. Government Accountability Office, U.S. Government Printing Office, Congressional Budget Office.  9:30 am. HT-2 Capitol. Committee on Appropriations Subcommittee on Appropriations.
• Solutions needed: Improper Payments Total $115 Billion in Federal Misspending. 2318 RHOB. Oversight and Government Reform Subcommittee on Government Organization, Efficiency, and Financial Management.

Relevant bills introduced:

• None.

Do you want to track transparency news? You can add our feed to your Google Reader, or view it on our Netvibes page.

As if independent expenditures by super PACs weren't enough, two major political players have notified the Federal Elections Commission of recent electioneering communications in states likely to be battlegrounds this fall.

Crossroads GPS reported spending $500,000 on ads against President Obama in Arizona, Michigan, Minnesota and Missouri, while  Planned Parenthood Action Fund Inc., spent $36,000 on ads in Florida and Michigan. Electioneering communication are TV or radio ads that refer to a candidate for office within 30 days of a primary or 60 days of a general election but stop short of calling for a vote for or against the candidate. The FEC must be notified of the ads and the amount spent on them but because Crossroads GPS and Planned Parenthood Action Fund are tax-exempt nonprofits, they do not have to disclose donors.

Crossroads GPS’s most recent ad “Entry Level” accuses Obama of focusing on failed projects such as solar panel maker Solyndra, while forgetting about those who have been laid off by those companies.

Planned Parenthood's 30-second spot applauds the president for protecting access to affordable birth control.

In the 2010 elections Crossroads--the group linked to GOP strategist Karl Rove spent more than $15 million in independent expenditures against Democrats and more than $1 million in electioneering communications -- according to OpenSecrets.  Sunlight's research indicates that Planned Parenthood Action Fund is a relatively new player in the field of electioneering communications, making their inaugural contributions, some $47,000, first during last year's special House election in New York on behalf of the winning candidate, Democrat Kathy Hochul. 

Read all about it

A former Commodity Futures Trading Commission (CFTC) commissioner has gone through the revolving door to the law and lobbying firm firm Patton Boggs, where he'll work as a senior policy advisor, the firm announced Tuesday.

"Regulatory agencies should be accessible to the businesses and industries they regulate," said Michael Dunn, the former commissioner, in a statement today. "I am excited to join the firm, and I look forward to helping clients navigate complex regulatory processes and agencies."

Appointed to the CFTC by former President George W. Bush, Dunn specialized in agricultural issues at the agency. His new employer is  a lobbying powerhouse that reported $36.7 million in lobbying income in 2011 alone. Patton Boggs has sent representatives to at least 30 meetings at federal financial agencies--24 of them at the CFTC--to lobby on the Dodd-Frank financial law, according to the Sunlight Foundation's Dodd-Frank meeting tracker.

Among Patton Boggs' recent meetings were several on behalf of the Wholesale Market Brokers Association, Americas, and its member groups. The organization faciliates swaps of derivatives, and considers itself a model for new entities, Swap Execution Facilities, created under Dodd-Frank to bring transparency to such transactions. 

Dunn himself does not appear in agency meeting logs on implementation of the Dodd-Frank law, which seems curious since he was at the commission through last October. A query to the CFTC press office about this issue was not returned by the time of this posting.

Read all about it

Lapidus is an Analytics Dashboard we developed in response to our desire to track metrics for all of our projects, whether they are web sites, APIs, mobile apps, etc. Sunlight has multiple projects that target different audiences and have different uses, but it is important for us to understand how all of these projects are used. Beyond that, we wanted to improve how we compared metrics across our projects -- while keeping in mind that not every possible comparison makes sense. With Lapidus we can view metrics across all of our projects in a single view, and when viewing aggregates across date ranges, Lapidus automatically color-codes certain metrics based on whether they increased or decreased from the previous period. Lapidus does not replace Google Analytics -- in fact it relies on GA for web metrics data -- but it does extend our ability to record and view additional metrics of our choosing.

This project was started by Jeremy Carbaugh (who named the project after a character from 'Lost'), who laid out the initial models for the metrics app with an eye toward flexibility. Ali Felski provided the design which also inspired some of the better features of the site (color-coding, sorting, etc.).

Capabilities

Lapidus is a Django project that consists of three fairly self-explanatory apps: metrics, loading, and dashboard (plus third-party apps and packages). The metrics app as you might expect stores the metrics data and metadata, and also contains the code for the API (using TastyPie). Lapidus's API gives projects the ability to push data into the system, so projects can actively contribute their own information. A Metric is defined by a combination of a Project and a Unit. The Unit carries some metadata about the metric: the category of metric ('web' and 'api' being the most common for Sunlight), the period (daily being the most common) and an observation_type, which determines what subclass of Observation gets recorded and provides clues to how the observation should be rendered.

The metrics app currently defines classes for CountObservation, ObjectObservation, and RatioObservation. The CountObservation is the basic workhorse for recording integer counts or various types. RatioObservation objects actually define a ratio between two existing observations, so we can record things like "bounce rates" and various averages and percentages. The ObjectObservation is potentially the most abstract, as it essentially stores JSON data. This is set to validate against a schema, currently a ranked list that contains a numeric value and possible a text label. This allows us to capture lists like "top pages" or anything of that sort. Since the underlying storage is JSON and the validation of the JSON occurs pre-save (using our own valedictory library), this class could be extended to validate and store other complex data types if the need arises.

The loading app consists of the management commands for loading data from various sources that we need to pull data from. Currently there are commands for Google Analytics, Facebook (shares), and custom endpoints. The loadendpoints and loadga commands rely on JSON configuration files to map what projects and metrics to load. The loadfacebook command simply looks at all projects that have a url defined on the model and submits a FQL request for link statistics. There is also a makeratios command for those times when you want to create an observation that is a ratio of two separate metrics. For example, to calculate "Average time on site per visit", we gather "time on site" and "visits" from Google Analytics (using loadga), and then run makeratios to create the desired "Average time on site per visit" observation. If a new source of metrics is to be added, doing so can be as simple as writing a new management command or configuring a project to push to Lapidus via the API.

The last app in Lapidus is the dashboard, the tool for determining how and when to display various metrics. The dashboard app, along with the templates, controls how the metrics are displayed. Given that various metrics are more than simple numbers ("time on site" is a count of time in seconds, for example), or that some metrics are not periodic but cumulative (Facebook shares), the dashboard had to handle different metrics depending on the various metadata. Of all the apps in Lapidus, this one probably took more time and consideration, and certainly is the first place to look for future development. There are a few models in dashboard that provide rudimentary assistance in controlling what gets displayed in the browser by allowing users to generate lists of projects and metrics. The dashboard gets the job done for all of the uses I was able to test, but there is certainly more that can be done to provide mechanisms for customizing and controlling the look of the data.

Technical Challenges

Metadata for Observations

While many metrics are counts for count's sake (Count Von Count would be proud), others may have a unit of measure such as time. Some metrics are ordered lists (top pages, top searches, etc.). Others are ratios that should be represented as decimals (pageview per visit) or percentages (bounce rate). There are certainly other types of metrics that weren't considered while writing this app. If you had a project oriented around sharing large datasets or media files, you might want to create a metric that records quantity of data downloaded per user, for example. Essentially, recording a numeric value is not always enough. Metrics require metadata to describe what the metric is. The CountObservation can handle metrics that can be represented as integers, which can handle many measurements, along with a little metadata. Google Analytics measures "time on site" in whole seconds (which python-googleanalytics returns as a float), so that fits in a CountObservation and the Metric object's Unit.observation_unit indicates that the number is a measure in seconds.

There are certainly additional descriptors that could be attached as metadata. The dashboard will color-code aggregate values for a date range based on whether they went up or down compared to the previous period. Really, the template will assign a class of .increased (green/good) or .decreased (red/bad), but currently the app assumes increases are good all of the time, which is not the case for certain metrics, like "bounce rate".

Lies, Damned Lies, and Analytics

If you've spent enough time working with Google Analytics data, you know that the numbers you see on Google Analytics can be a bit… slippery. Various explanations can be found on the forums, that all of the various servers gathering analytics don't always report back on time is one of the more reasonable-sounding explanations. Scott Stadum, our resident analytics analyzer, has seen different numbers from the front end versus the API as well as when checking the same observation on a different day. A search on the forums will turn up others who have seen the same issue. There are surely legitimate reasons to revise data, particularly when seeking the most accurate measures. However, not knowing when or how often these changes are made (and if there is a point when the number becomes canonical) makes it difficult to know when a tool like Lapidus should go back and revise it's recorded numbers. Moving forward, this is something that may cause new headaches, but at least the numbers can be reimported with a simple management command.

Tweets per site

There were a few other metrics we wanted to capture, amongst them the number of tweets about a site. There are analytics tools that capture and calculate this, and Google Analytics has recently started tracking tweets and displaying them in their front-end. However, their calculations are not currently available via their API. The Twitter API has an unofficial endpoint where you can uncover the the 'tweet count' for a particular URL, but they (to my knowledge) do not calculate a value for an entire domain. To calculate this intelligently, Lapidus would need to generate a site map (filtering out non-tweetable pages like API URLs) for a project's domain and then query Twitter for each URL. Google is already doing this heavy lifting (after acquiring a few social analytics companies in 2011), they simply have not yet exposed this social data as part of their API. There is a feature request ticket on their issue tracker, and the social engagement features are relatively new to Google Analytics, so it seems likely that this will become available in the near future.

Room for improvement

There are already some features we know we would like to add or update, amongst them:

• Update to Google Analytics' new Core Reporting API. On December 5, Google announced an update (and name change) to their Data Export API. Version 3.0 of the newly minted Core Reporting API returns JSON rather than XML, and requires that applications be registered and use OAuth 2.0 for requests. This announcement was made fairly late in the process for developing Lapidus, but updating to Core Reporting API v3.0 is a priority moving forward.
• Test new metrics. So far every metric used to test this app has been a daily-recorded value or a cumulative value with no period, so the daily time period is the one supported in the dashboard. Some of this is simply writing new (or updated) views, but there are also considerations to make about how different periods will display. When does it make sense to display hourly measurements next to daily? or weekly? At some point this will become an issue that needs to be addressed.
• More metadata. Beyond the numbers and lists we record, it would be great to provide ways to define more qualitative analyses. What is a good or bad number when talking about bounce rate or number of downloads or any other metric?
• Build admin tools for configuration and customization. In order to support more customization options, it makes sense to build out an admin that allows the customization of views and observation metadata in the admin web interface. Currently some of the customization is in the admin (creating Project, Unit, and Metric objects), while some pieces are in JSON files (connecting Project's to Google Analytics, for example). A truly flexible dashboard will allow site administrator's to handle configuration, define additional metadata (used to provide additional options used in rendering templates), and control view output via the admin interface. It will be important to find the right balance between flexibility and simplicity.

Moving forward

Lapidus seeks to address some of Sunlight's analytics reporting needs while remaining flexibility enough for others to adapt to their metrics needs. We've put our code on Github; we hope those with similar needs will find it useful.