Showing posts with label data mining. Show all posts
Showing posts with label data mining. Show all posts

Wednesday, April 10, 2013

How 86 Journalists in 46 Countries Worked Together to Expose Tax Shelters of Politicians, Fundraisers. and Celebrities from Over 170 Countries

This is amazingly cool - I look forward to reading what this data mining reveals about the insane amount of money hidden in off-shore, untaxed accounts.

This excellent summary of the work that went into exposing this information comes from the Neiman Journalism Lab at Harvard University.


Intercontinental collaboration: How 86 journalists in 46 countries can work on a single investigation


Over 2.5 million files analyzed by a global team of journalists reveal financial information about politicians, fundraisers. and celebrities from over 170 different countries.

By Caroline O’Donovan

On Thursday morning, the International Consortium of Investigative Journalists— a project of the Center for Public Integrity — will begin releasing detailed reports on the workings of offshore tax havens. A little over a year ago, 260 gigabytes of data were leaked to ICIJ executive dIrector Gerard Ryle; they contained information about the finances of individuals in over 170 countries.

Ryle was a media executive in Australia at the time he received the data, says deputy director Marina Walker Guevara. “He came with the story under his arm.” Walker Guevara says the ICIJ was surprised Ryle wanted a job in their small office in Washington, but soon realized that it was only through their international scope and experience with cross border reporting that the Offshore Project could be executed. The result is a major international collaboration that has to be one of the largest in journalism history.

“It was a huge step. As reporters and journalists, the first thing you think is not ‘Let me see how I can share this with the world.’ You think: ‘How can I scoop everyone else?’ The thinking here was different.” Walker Guevara says the ICIJ seriously considered keeping the team to a core five or six members, but ultimately decided to go with the “most risky” approach when they realized the enormous scope of the project: Journalists from around the world were given lists of names to identify and, if they found interesting connections, were given access to Interdata, the secure, searchable, online database built by the ICIJ.

Just as the rise of information technology has allowed new competition for the attention of audiences, it’s also enabled traditional news organizations to partner in what can sometimes seem like dizzyingly complex relationships. The ICIJ says this is the largest collaborative journalism project they have ever organized, with the most comparable involving a team of 25 cross border journalists.

In the end, the Offshore Project brings together 86 journalists from 46 countries into an ongoing reporting collaboration. German and Canadian news outlets (Süddeutsche Zeitung, Norddeutscher Rundfunk, and the CBC) will be among the first to report their findings this week, with The Washington Post beginning their report on April 7, just in time for Tax Day. Reporters from more than 30 other publications also contributed, including Le Monde, the BBC and The Guardian. (The ICIJ actually published some preliminary findings in conjunction with the U.K. publications as a teaser back in November.)

“The natural step wasn’t to sit in Washington and try to figure out who is this person and why this matters in Azerbaijan or Romania,” Walker Guevara said, “but to go to our members there — or a good reporter if we didn’t have a member — give them the names, invite them into the project, see if the name mattered, and involve them in the process.”

Defining names that matter was a learning experience for the leaders of the Offshore Project. Writes Duncan Campbell, an ICIJ founder and current data journalism manager:
ICIJ’s fundamental lesson from the Offshore Project data has been patience and perseverance. Many members started by feeding in lists of names of politicians, tycoons, suspected or convicted fraudsters and the like, hoping that bank accounts and scam plots would just pop out. It was a frustrating road to follow. The data was not like that.
The data was, in fact, very messy and unstructured. Between a bevy of spreadsheets, emails, PDFs without OCR, and pictures of passports, the ICIJ still hasn’t finished mining all the data from the raw files. Campbell details the complicated process of cleaning the data and sorting it into a searchable database. Using NUIX software licenses granted to the ICIJ for free, it took a British programmer two weeks to build a secure database that would allow all of the far-flung journalists not only to safely search and download the documents, but also to communicate with one another through an online forum.

“Once we went to these places and gathered these reporters, we needed to give them the tools to function as a team,” Walker Guevara said.

Even so, some were so overwhelmed by the amount of information available, and so unaccustomed to hunting for stories in a database, that the ICIJ ultimately hired a research manager to do searches for reporters and send them the documents via email. “We do have places like Pakistan where the reporters didn’t have much Internet access, so it was a hassle for him,” says Walker Guevara, adding that there were also security concerns. “We asked him to take precautions and all that, and he was nervous, so I understand.”

They also had to explain to each of the reporting teams that they weren’t simply on the lookout for politicians hiding money and people who had broken the law. “First, you try the name of your president. Then, your biggest politician, former presidents — everybody has to go through that,” Walker Guevara says. While a few headline names did eventually appear — Imelda Marcos, Robert Mugabe — she says some of the most surprising stories came from observing broader trends.

“Alongside many usual suspects, there were hundreds of thousands of regular people — doctors and dentists from the U.S.,” she says, “It made us understand a system that is a lot more used than what you think. It’s not just people breaking the law or politicians hiding money, but a lot of people who may feel insecure in their own countries. Or hiding money from their spouses. We’re actually writing some stories about divorce.”

In the 2 million records they accessed, ICIJ reporters began to get an understanding of the methods account holders use to avoid association with these accounts. Many use “nominee directors,” a process which Campbell says is similar to registering a car in the name of a stranger. But in their post about the Offshore Project, the ICIJ team acknowledges that, to a great extent, most of the money being channeled through offshore accounts and shell companies is actually not being used for illegal transactions. Defenders of the offshore banks say they “allow companies and individuals to diversify their investments, forge commercial alliances across national borders, and do business in entrepreneur-friendly zones that eschew the heavy rules and red tape of the onshore world.”

Walker Guevara says that, while that can be true, the “parallel set of rules” that governs the offshore world so disproportionately favor the elite, wealthy few as to be unethical. “Regulations, bureaucracy, and red tape are bothersome,” she says, “but that’s how democracy works.”

Perhaps the most interesting question surrounding the Offshore Project, however, is how do you get traditional shoe-leather journalists up to speed on an international story that involves intensive data crunching. Walker Guevara says it’s all about recognizing when the numbers cease to be interesting on their own and putting them in global context. Ultimately, while it’s rewarding to be able to trace dozens of shell companies to a man accused of stealing $5 billion from a Russian bank, someone has to be able to connect the dots.

“This is not a data story. It was based on a huge amount of data, but once you have the name and you look at your documents, you can’t just sit there and write a story,” says Walker Guevara. “That’s why we needed reporters on the ground. We needed people checking courthouse records. We needed people going and talking to experts in the field.”

All of the stories that result from the Offshore Project — some of which could take up to a year to be published — will live on a central project page at ICIJ.org. The team is also considering creating a web app that will allow users to explore some (though probably not all) of the data. In terms of the unique tools they built, Walker Guevara says most are easily replicable by anyone using NUIX or dtSearch software, but they won’t be open sourced. Other lessons from the project, like the inherent vulnerability of PGP encryption and “other complex cryptographic systems popular with computer hackers,” will endure.

“I think one of the most fascinating things about the project was that you couldn’t isolate yourself. It was a big temptation — the data was very addictive,” Walker Guevara says. “But the story worked because there was a whole other level of traditional reporting that was going and checking public records, going and seeing — going places.”

Photo by Aaron Shumaker used under a Creative Commons license.

Monday, February 18, 2013

The River of Myths: Hans Rosling Visualizes the Incredible Progress of the “Developing World” in Under 3 Minutes


Via Open Culture, the internet's curators of cool - this video from Hans Rosling is pretty amazing.

In Under Three Minutes, Hans Rosling Visualizes the Incredible Progress of the “Developing World”

February 18th, 2013


Hans Rosling knows how to make a concise, powerful point. His mastery of statistics and visual aids doesn’t hurt. Behold, for instance, the Karolinska Institute Professor of International Health visualizing the health of 200 countries over 200 years with 120,000 data points. His ability to condense vast amounts of information into short bursts while providing the widest possible context for his points naturally endears him to the TED audience, which values counterintuitive intellectual impact delivered with the utmost succinctness. We previously featured a TED Talk from wherein the excitable professor explains world population growth and prosperity with props bought at IKEA. (The man comes from Sweden, after all. One must represent.) Now, on Bill Gates’ Youtube channel, you can watch Rosling’s shortest and slickest video yet: “The River of Myths.”

Opening with a visualization of 1960′s world child mortality numbers graphed against the number of children born per woman, Rosling uses his signature method of statistical-animation showmanship to explode myths about the potential of developing nations. We see that, as a country’s wealth rises, its health rises; as its health rises, its child mortality drops; and as its child mortality drops, so does its number of children born per woman, which leads to a sustainable overall population size. He then examines the separate regions of Ethiopia, formerly a developmental laggard, showing that the capital Addis Ababa ranks reproductively among the developed nations, while only remote regions lag behind. “Most people think the problems in Africa are unsolvable, but if the poorest countries can just follow the path of Ethiopia, it’s fully possible that the world will look like this by 2030.” We then see a projection of all the world’s nations clustered in the small-family, low-mortality corner of the graph. “But to ensure this happens, we must measure the progress of countries. It’s only by measuring we can cross the river of myths.” Have you heard a more powerful argument for the usefulness of statistics lately?

Related content:

Colin Marshall hosts and produces Notebook on Cities and Culture and writes essays on literature, film, cities, Asia, and aesthetics. He’s at work on a book about Los Angeles, A Los Angeles Primer. Follow him on Twitter at @colinmarshall.

Thursday, July 05, 2012

Ban on Drugs Interferes With Brain Research, So Scientists Get Creative


Reuters ran an article at the end of May that featured David Nutt - professor of neuropsychopharmacology at Imperial College London and a former chief adviser on drugs to the British government - who argued that the international prohibition on psychedelics and other mind-altering drugs over the last 50 years or so has been damaging to brain research and has had "perverse" consequences for our efforts to solve the mysteries of various mental illnesses.

It's important than researchers at his level speak out on this topic - things are shifting slowly in this area, with a lot of new research underway, but it's still exceedingly difficult to get FDA approval in the U.S.

The restrictions on research have forced investigators to be creative. That is exactly what we find in an article from MIT's Technology Review on data mining the internet for users' first-person descriptions of the effects of psychedelic drugs. The obvious resource they tap is erowid.org, a well-known and popular source of user-generated information about the effects of all kinds of psychoactive substances. Very interesting.

Drug bans hamper brain research, says neuroscientist

The British government's former chief drugs adviser, David Nutt, reacts as he speaks during a news conference announcing the formation of the Independent Scientific Committee on Drugs, in London January 15, 2010. REUTERS/Suzanne Plunkett
 LONDON | Thu May 31, 2012 
 
(Reuters) - Bans on drugs like ecstasy, magic mushrooms and LSD have hampered scientific research on the brain and stalled the progress of medicine as much as George Bush's ban on stem cell research did, a leading British drug expert said on Thursday.

David Nutt, a professor of neuropsychopharmacology at Imperial College London and a former chief adviser on drugs to the British government, said the international prohibition of psychedelics and other mind-altering drugs over the past half century has had damaging and "perverse" consequences.
"When a drug becomes illegal, conducting experimental research on it becomes almost impossible," Nutt told reporters at a briefing in London ahead of the publication of his new book "Drugs - without the hot air".

He compared the situation with that in stem cell research under former U.S. President George W. Bush, who banned any new embryonic stem cell studies from 2001 to 2009 - a move many scientists consider held the field back for years.

Nutt said the problem with the current approach to drugs policy globally, which is centered on the banning of substances thought to be most harmful, "is that we lose sight of the fact that these drugs may well give us insights into areas of science which need to be explored and they also may give us new opportunities for treatment."

"Almost all the drugs which are of interest in terms of brain phenomena like consciousness, perception, mood, psychosis - drugs like psychedelics, ketamine, cannabis, magic mushrooms, MDMA - are currently illegal. So there's almost no (scientific)work in this field," Nutt said.

Nutt last year conducted a small human trial to study the effects of psilocybin, the active ingredient in magic mushrooms, on the brain.

Contrary to scientists' expectations, the study found psilocybin doesn't increase but rather suppresses activity in areas of the brain linked to depression, suggesting the drug might be a useful treatment for the debilitating condition.

Nutt said he was forced to "jump through hundreds of hoops" to be able to conduct the study, having to comply with a level of complex, expensive and time-consuming security and regulation that would put most scientists off.

WHAT DRUGS ARE AND WHAT THEY DO

The professor, who was sacked in 2009 in a high-profile row with the British government after he compared the risks of smoking cannabis with those of riding a horse, said he was driven to write the book in the hope of improving understanding of drugs - both legal and illegal, medicinal and recreational.

"There is almost no one in society who doesn't take drugs of some sort. The choices you make in your drug-taking are driven by a complex mixture of fashion, habit, availability and advertising," he said.
"If we understand drugs more, and have a more rational approach to them, we will actually end up knowing more about how to deal with drug harms."

Published on Thursday, the book seeks to explore the science of what a drug is and how it works.

It discusses whether the "war on drugs" did more harm than good - Nutt thinks it did.

And it explores why Britain's Queen Victoria took cannabis - apparently her physician J.R. Reynolds wrote a paper in the Lancet medical journal saying that "when pure and administered carefully, it (cannabis) is one of the most valuable medicines we possess". He prescribed it to the monarch to help her with period pains and after childbirth.

The book also has chapters on why people take drugs now, how harmful they are, where and whether the danger lines should be drawn between legal drugs like tobacco and alcohol, and illegal ones like cannabis and magic mushrooms.

Nutt doesn't dispute that drugs are harmful, but he takes issue with what he says are un-scientific decisions to ban one, like cannabis, while allowing another, like alcohol, to be freely and cheaply available on supermarket shelves.

"Drugs are drugs. They may differ in terms of their brain effects, but fundamentally they are all psychotropic agents," he said. "And it's arbitrary whether we choose to keep alcohol legal and ban cannabis, or make tobacco legal and ban ecstasy. Those are not scientific decisions they are political, moral and maybe even religious decisions."

(Reporting by Kate Kelland, editing by Paul Casciato)
 * * * * * * *

Here is the article on data mining for knowledge of drug effects.

Psychedelic Drug Research and the Data-Mining Revolution

The Web is filled with users' descriptions of the effects of psychedelic drugs. Now neuroscientists are using data-mining techniques to quantify the effects of these drugs on human consciousness.
 

One of the most mysterious problems in neuroscience is the link between brain chemistry and consciousness. How do changes in our neurochemistry influence our perception of the real world? 
This question is hard to tackle for the obvious reason that experiments on humans are notoriously difficult to perform. Not only are the variables hard to pin down but changing them with psychoactive drugs under controlled conditions is fraught with practical, ethical, and moral dilemmas.

That's why the majority of work examining the role of psychoactive drugs on neuropharmacological signaling mechanisms has been done on rats.

But there's a revolution afoot. Today, Jeremy Coyle at the University of California Berkeley and a couple of pals say they've found a new way to study the role of psychoactive drugs on human perception.

These guys point out that in the contrast to the small amount of formal scientific literature in this area, there are large volumes of narrative descriptions of the effects of drugs posted on the web. Their idea is to mine these descriptions using machine learning techniques to identify common features which would allow a quantitative comparison of their effects.

The obvious place to start such an endeavour is a website called erowid.org, which is a well known and popular source of user generated information about the effects of all kinds of psychoactive substances.

Coyle and co confine their investigations to ten drugs ranging from
3,4‐methylenedioxymethamphetamine, better known as ecstacy, and lysergic acid diethylamide, or LSD, to less well known drugs such as N,N‐dipropyltryptamine, sometimes called The Light,  and
2,5‐dimethoxy‐4‐ethylphenethylamine which has the street name Europa.

They collected 1000 narrative reports on these drugs and mined the text for common words, while screening out some words that are common to more than five drugs.

Having identified signature words, they then tested their hypothesis by seeing whether the results could be used to accurately predict which drug the reports referred to.

It turns out that some drug reports are much easier to classify than others. Ecstasy reports tends to use words such as “club”, “hug”, “rub” and “smile", which reflect the social setting in which the drug is often used and the feelings of love and friendliness the drug seems to produce.


Read the whole article.

You can read the whole paper online for free - here is the abstract.



Jeremy R. Coyle
David E. Presti
Matthew J. Baggott

ABSTRACT
 
BACKGROUND: Psychedelic drugs facilitate profound changes in consciousness and have potential to provide insights into the nature of human mental processes and their relation to brain physiology. Yet published scientific literature reflects a very limited understanding of the effects of these drugs, especially for newer synthetic compounds. The number of clinical trials and range of drugs formally studied is dwarfed by the number of written descriptions of the many drugs taken by people. Analysis of these descriptions using machine‐learning techniques can provide a framework for learning about these drug use experiences.

METHODS: We collected 1000 reports of 10 drugs from the drug information website Erowid.org and formed a term‐document frequency matrix. Using variable selection and a random‐forest classifier, we identified a subset of words that differentiated between drugs.
 
RESULTS: A random forest using a subset of 110 predictor variables classified with accuracy comparable to a random forest using the full set of 3934 predictors. Our estimated accuracy was 51.1%, which compares favorably to the 10% expected from chance. Reports of MDMA had the highest accuracy at 86.9%; those describing DPT had the lowest at 20.1%. Hierarchical clustering suggested similarities between certain drugs, such as DMT and Salvia divinorum.

CONCLUSION: Machine‐learning techniques can reveal consistencies in descriptions of drug use experiences that vary by drug class. This may be useful for developing hypotheses about the pharmacology and toxicity of new and poorly characterized drugs.