About

Feeds

 

〰 Tidal, Archiloque's feed reader

AltDevBlogADay“Refactor”

Refactoring is defined as a “disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior” by Martin Fowler.

But its everyday usage takes a very different meaning. We use the word refactor to everything from the original meaning, to a complete side-by-side rewrite of a barely-functional system. How can you use the same word to refer to something that does not change external behavior, with building an entirely new replacement system?

More than once, I’ve elicited sighs for my efforts to clarify the language we use. But when you have people that use wildly different vocabularies- artists, programmers, project managers- this is of paramount importance. So the fact that we say ‘refactor’ to meaning any type of rewriting of code or functionality irks me.

So recently I’ve begun a dictionary in my head for the type of tasks we do.

Refactor: The original and ‘precise’ meaning of restructuring a module without altering external behavior, up to a rewrite of parts of a larger system, that may change internal behavior of the system (including the external behavior of large internal components). While I’d love to keep only the more limited original meaning, when we’re dealing with large legacy codebases that often have zero tests, the original meaning doesn’t apply often enough.

Renovate: Rewriting a system so that its external behavior changes considerably but it still fulfills the same purpose (obviously). I name it as such because it is like renovating a building- the exterior and interior may greatly change, but what goes on inside may stay the same. Similar to ‘Rewrite’ but generally applies to a smaller (module/system) scale. An example would be, if you have a library for dealing with source control, and you no longer like the API. So you greatly change how the module works, and it still fulfills its fundamental purpose of dealing with source control.

Recycle: Writing a replacement system side-by-side with the old system, using components from the old system (by either referencing or copy/pasting), with the goal that once the new system is working, the old system will be shut down. The goal is a replacement that is easier to use but fulfills similar requirements. An example would be replacing legacy procedures for data transformations, that may have a lot of imperative code and poor reuse. You would replace the system with something better written, often taking chunks of logic from the old system, or writing tests that verify it produces the same results, then hook up calling code to use the new system, then delete the old one entirely.

Rewrite/Rebuild: When a system is to be written from the ground up, using the original implementation as an example or in a prototype role only, resulting in a system that fulfills the business requirements of the original system but does not necessarily preserve anything else about it. Similar to ‘Renovating’ but generally applies to a larger (application) scale. An example would be, if you have a website/game feature that you want to replace, you’d build the new one to fulfill the same business requirement (we need a website/some feature), but the features may be completely different.

So that’s my personal language right now. My hope is that it will expand to my team and then out from there. I don’t know if I’ll refine it much more- too granular and it would become unwieldy. Do you have suggestions or a language of your own?

Ars TechnicaVita's "augmented reality" games seem to miss meaning of "augmented"

A few days after I received my PlayStation Vita review unit in the mail, I got a set of six plastic cards with boxy black and white patterns on them. The free "augmented reality" games that rely on those cards were made available for download on the PlayStation Network today, and while the games definitely take place on a backdrop of reality, I'm not really sure how much the real world is being "augmented" through the Vita.

Read the rest of this article...

Read the comments on this post


Ars TechnicaiPhone and Android apps now required to have privacy policies

The makers of the most widely used mobile app stores have agreed to comply with a California law requiring mobile apps that collect personal information to have a privacy policy. California Attorney General Kamala Harris announced the agreement today with Apple and Google, which run the two most popular mobile app stores, as well as Amazon, HP, Microsoft, and Research In Motion.

"These platforms have agreed to privacy principles designed to bring the industry in line with a California law requiring mobile apps that collect personal information to have a privacy policy," Harris's office said in a press release. "The majority of mobile apps sold today do not contain a privacy policy."

The agreement doesn't place restrictions on what types of data app makers may collect. But app makers must describe "how personal data is collected, used and shared," and make their privacy policies easily found by users. App store listings will contain either the text of the privacy policy or a link to the policy.

There have been several controversies over mobile app privacy, and one of the most recent centered on the social network Path accessing and uploading iPhone users' contact databases without permission. Harris noted that a Wall Street Journal report last year found "that 45 of the top 101 apps did not provide privacy policies either inside the application or on the application developer’s website," despite the fact that most of the mobile apps were transmitting a phone's unique device ID or location "to other companies without users' awareness or consent." Some apps were also transmitting the user's age, gender, and other personal details.

Read the comments on this post


Danger RoomBlasts to the Head ‘Primed’ Brains for PTSD, Study Says

Photo: U.S. Air Force

The “signature wounds” of the wars in Iraq and Afghanistan — post-traumatic stress disorder and traumatic brain injury — are both rooted in traumatic events. Until recently, though, military docs mostly treated them as two different health problems: one physical, the other psychological. That approach might be poised to change, thanks to a new study, which shows that injuries to a specific part of the brain “primed” it for PTSD’s psychological ailments.

Post-traumatic stress disorder is widely known as the psychological condition that’s followed soldiers home from Iraq and Afghanistan. The connection between war and PTSD is simple enough: Soldiers undergo a traumatic experience, if not several, overseas. Those traumas stay with them, and seem to have a profound impact on their stress hormones and brain chemistry. The result? Symptoms like nightmares, paranoia and angry outbursts.

In comparison, traumatic brain injuries (TBIs) seem extremely different. These injuries are caused by a physical trauma — an IED attack, for example — that actually rattles the brain inside the skull. Subsequent brain damage can cause everything from vomiting and headaches to long-term loss of sensation and speech impediments.

Scientists have known for a while that TBIs and PTSD are connected. One 2008 study concluded that 44 percent of personnel with a TBI also suffered from PTSD, compared to 9 percent of those without physical injury. Of course, the link seems obvious: It follows that driving a Humvee that’s suddenly blown to pieces will rattle the skull and also trigger psychological distress.

But researchers now suspect that the link goes even further: They’ve concluded that the physical blow from a TBI changes a key part of the brain, making a soldier more at risk of developing PTSD in the future.

Scientists at the University of California at Los Angeles, led by Dr. Maxine Reger, this week published a study that uses rats to examine the relationship between TBI and PTSD.

They first divided the animal subjects into two groups. The researchers used physical force to cause TBIs among the rats in one group. Those in the other group were left unscathed.

Then the team waited two days before exposing the rats to fearful experiences. The point of waiting was to separate the physical trauma (the TBI) from the psychological. Researchers wanted to know whether, days later and during an entirely different experience, a TBI would have any impact on PTSD risk.

Lo and behold, rats in the TBI group did react differently to fearful stimulus. In fact, they exhibited “inappropriately strong fear,” according to Dr. Michael Fanselow, one of the researchers involved in the study. Rats with healthy noggins, however, exhibited more appropriate reactions.

“It was as if the injury primed the brain for learning to be afraid,” he said in a statement.

At the core of the finding is one brain region, called the amygdala. Scientists already know that this tiny bundle of neurons is extremely vulnerable to damage during a brain-rattling event that causes a TBI. The amygdala is also one of the most important brain areas where PTSD is concerned, because it regulates fear response.

After their experiments, the team analyzed brain tissue from the amygdala of several rats. Among rats afflicted with a TBI, the amygdala had significantly more receptors for neurotransmitters that are involved in the learning process. In other words, a TBI somehow causes these receptors to multiply, meaning that there are more of them available to be activated by neurotransmitters. So when a person is exposed to a scary event, their amygdala is, oddly enough, more capable of learning fear.

In a human context, the study’s findings (roughly) suggest the following: If two soldiers are exposed to the same psychologically scarring event, the soldier who suffered a TBI last month would be more likely to develop PTSD than his colleague.

Of course, it’s not that simple. Scientists already know that all sorts of other factors, from genetics to childhood environment, affect a soldier’s risk of developing PTSD. And even if TBIs are a risk factor, they aren’t the only one — not everyone who gets PTSD, whether soldier or civilian, also suffered a traumatic brain injury sometime in the past. Likewise, not all TBI sufferers eventually end up with PTSD.

That said, the study could break new ground in the Pentagon’s efforts to treat, diagnose and prevent PTSD and TBI. Thus far, the military hasn’t had much luck unraveling either one. Figuring out how and where the two illnesses are tangled together might be a good place to start.


LWN.netABS/ELC videos available
The Linux Foundation has posted videos of the talks given at the 2012 Android Builders Summit and Embedded Linux Conference. While most of the talks are up, evidently they are still working on some of the videos; the remaining talks should appear shortly.
Daring FireballFun With Charts, Fox News Edition

Almost comically shameless.

 ★ 
Daring FireballPolar Bear Farm’s One-Day Sale to Help Christchurch, New Zealand

Must-read letter from Layton Duncan, founder of the software shop Polar Bear Farm in Christchurch New Zealand, on the one-year anniversary of the earthquake:

Our city has been destroyed. It’s hard to find a natural disaster anywhere which has a larger effect on a country than this earthquake is having here. In relative terms, the effect on New Zealand is equivalent to around 8 Hurricane Katrinas, or around 3 2011 Sendai earthquake and tsunamis. Christchurch needs serious help to recover from this once in 10,000 year event. […]

On Wednesday 22nd of February 2012, all our apps will be reduced in price. 100% of the proceeds of all sales for the day will go into seeding the formation of a charitable trust with the explicit purpose of kickstarting the creation of a built environment for a safe, vibrant, sustainable downtown Christchurch people can inhabit again.

Great apps, great prices, great cause.

 ★ 
EgeablogRejoindre le Groupe des Plans Nucléaires (GPN) : une ineptie

Encore une fois, on me passe un texte où un esprit, "libre" et délié et innovant, balance l'idée géniale : rejoignons le Groupe des Plans Nucléaires à l'OTAN. Ce qui n'est d'une erreur, me semble-t-il. Et puisque c'est une idée "innovante", discutons-la : pas de thèse valable sans une bonne antithèse.

source

L'idée de réintégrer le GPN de l'Otan est régulièrement agitée par les "brillants" esprits généralistes qui ne connaissent pas grand chose à l'Otan. Chaque année, un brillant plein d'avenir jette l'idée dans tel rapport plus ou moins confidentiel. Mais je subodore que ce doit être également écrit dans des copies de l'école de guerre et autres lieux où on dit aux gars : vous devez terminer par des propositions. Et comme les gars ne connaissent pas grand chose, ils s'attachent aux organigrammes, voient qu'on n'a pas rejoint le GPN, et hop!, passez muscade, v'là que jte l'propose, le vorace y sera content, ça au moins c'est original et brillant.

De même, on voit pas mal d'universitaires qui ne connaissent l'Alliance que de l'extérieur et qui se précipitent sur un organigramme et croient qu'en modifiant une position dans le dit organigramme, tout va changer, y compris politiquement : quiconque a pratiqué n'importe laquelle grande organisation sait que les organigrammes sont beaux, mais que ce ne sont pas eux qui font marcher un système (voir billet ici). Et bizarrement, c'est la même chose dans cette grande machine politique et "intergouvernementale" qu'est l'OTAN. Ou, dans ce cas précis, l'Alliance puisque, faut-il le préciser, le GPN appartient au dispositif de l'alliance (le côté politique) et non au dispositif militaire, l'organisation proprement dite.

Ainsi donc, et à moins qu'il y ait eu un brusque changement de position validé à la fois par le quai d'Orsay, l'EMA et l'EMP, mais qui alors n'a pas été rendu public et s'inscrirait en total porte-à-faux avec la doctrine et la constante pratique française, réaffirmée notamment à Strasbourg Kehl et à Lisbonne, cette proposition n'est pas valide aujourd’hui pour cinq raisons.

source

En effet, une telle proposition, c'est :

1/ ne rien connaître à l'Otan (et donc ne pas savoir que le GPN ne sert pas à grand chose). En effet, il fut créé en 1966 après notre départ tout d'abord pour qu'il y ait un lieu "politique" où discuter à 15 (sans donc les Français) ; et qu'ensuite il s'inscrivait dans une grande dispute transatlantique portant sur le couplage nucléaire, précisément sur la question de la riposte graduée, et qu'il s'agissait pour les Américains de montrer aux alliés qu'ils avaient leur mot à dire en matière nucléaire, contrairement à ce que racontaient ces idiots de Français et que bien sûr, bien sûr, avant d'engager leurs bombes nucléaires en Europe, ils consulteraient les alliés. Les choses ont, le sait-on, un peu changé depuis 1966 : un tout petit peu ... ! Et si l'Alliance demeure une alliance "nucléaire", c'est d'abord grâce aux armes américaines, les fameuses B 61 qui constituent, pour le coup, un enjeu réel de Chicago. Et d'ailleurs la source d'une certaine dissension américano-allemande.

2/ c'est ne rien connaître non plus à la réalité allemande : allez dire à un Allemand de partager le nucléaire, essayez seulement ! cela fait vingt ans qu'on leur propose, qu'ils refusent, et au dernier sommet de Lisbonne, M. Westerwelle en a fait un objet de dissension franco-allemand, provoquant une crise diplomatique qui a raidi les Français comme des puces bretonnes en manque de chouchen et obligé Mme Merkel de venir calmer les choses. Ne pas en douter, ils risquent de remettre le couvert à Chicago, car pour eux, grâce à la sainte DAMB, plus besoin de cette arme horrible et sale et pas morale. Et ce serait le moment de rejoindre le GPN et de partager notre arme avec les autres Européens ? avec les Allemands ??

3/ c'est ne rien comprendre à ce que veulent les Américains, qui seront trop contents de nous retirer ce qu'ils considèrent comme une prolifération indue des années 60 et qui y verrons une belle revanche posthume sur De Gaulle. Habile !

4/ c'est ne pas avoir compris la logique profonde du traité FR-UK qui est d'abord nucléaire, ne l'oublions pas.

5/ c'est enfin ne pas avoir le premier centime d'ancien franc de sens politique et ne pas comprendre que le nucléaire, ça ne se partage pas et que c'est absolument essentiel à la géopolitique de la France : l'arme grâce à laquelle on ne revivra pas juin 1940 et la saignée de 14-18. Mais là, il ne s'agit pas de notre place dans l'alliance, il s'agit de la place du nucléaire dans notre stratégie nationale.

Bref, le nucléaire, avant d'être un dogme, est quelque chose qu'il faut "penser". Et si on pensait un peu, avant de faire des propositions farcies sur le nucléaire ?

O. Kempf

Daring FireballNightline: Apple’s Chinese Factories (Requires Flash)

ABC News’s exclusive behind-the-scenes look at Foxconn’s Apple product factories. I thought it was both fair and fascinating. Absolutely worth watching.

 ★ 
The GitHub BlogGaug.es for Android

We're proud to announce that Gaug.es for Android is now available in the Android Market! The app provides a quick overview of your traffic, top content, top referrers, and the live AirTraffic for each of your gauges.

All versions of Android from Froyo (Android 2.2) to Ice Cream Sandwich (Android 4.0) are supported.

We've started small but with the same love and attention that you've gotten from the rest of the Gaug.es family of apps. You'll get a quick look dashboard with all of your gauges, list views of your daily traffic, most viewed content, top web referrers and be able to watch visitors from all over the globe hit your websites as it happens with AirTraffic.

If you have a Gaug.es account, head on over to the Android Market and install the app - it's completely free. If you haven't tried Gaug.es yet, sign up for a free 7-day trial and see if Gaug.es is right for you!

The app is free in more than one sense of the word, as it's also open-source under the Apache 2.0 license and you can grab the source here.

Gaug.es for Android is built on top of the awesome Gaug.es API as well as several great projects for Android development that are also available on GitHub:

Ars TechnicaApple's latest sandboxing deadline delay signals moving goalpost for devs

Apple has given developers yet another few months to implement application sandboxing for OS X apps, a security feature brought over from iOS: the deadline is now June 1, 2012. While the intent of sandboxing is to prevent hacked apps from taking over a user's system, however, the sandbox design inherently limits functionality that users and developers have come to expect on the desktop. Apple's changes and delays to sandboxing requirements have also created a situation where the sandboxing goalpost keeps moving while developers continue to push Apple to improve its design.

Read the rest of this article...

Read the comments on this post


Ars TechnicaT-Mobile seeks to block Verizon spectrum purchase

The nation's fourth-largest cellular firm has asked the Federal Communications Commission to block a spectrum acquisition by its largest competitor. T-Mobile argues that allowing Verizon to purchase more spectrum would make it too difficult for smaller wireless firms to build next-generation networks of their own.

The spectrum under dispute was acquired in an auction by a coalition of cable companies led by Comcast and Time Warner in 2006. But the cable firms have apparently decided they don't want to be in the wireless business after all. In December, Verizon Wireless announced plans to buy the spectrum, which is in the AWS band, for $3.6 billion.

Read the rest of this article...

Read the comments on this post


Planet IntertwinglyTagging mechanisms and strategies part 3: Taxonomy and folksonomy
Should a tag namespace be a top-down taxonomy or a bottom-up folksonomy? My answer is: both. In recent months, as I curate calendar hubs for selected cities, I’ve been working toward an approach that harmonizes the two styles. Principle: Top-down and bottom-up In the elmcity context, the most important taggable object is the calendar feed. That’s because when you can characterize a whole feed with a tag, all the events in that feed inherit the tag. The primary sources of taggable feeds are Eventful, Upcoming, Facebook, Meetup, and EventBrite. I call them taggable because, while some of these services tag individual events, none tag feeds based on venues (Eventful, Upcoming) or Pages (Facebook) or groups (Meetup) or organizers (EventBrite). Assigning feed-level tags is an editorial exercise for the curator. To these sources I add as many standalone iCalendar feeds as I can find. For Boston and Seattle, the results add up to lists of over 600 tagged iCalendar feeds. Here’s a table of the current list of tags for Boston, the current list for Seattle, and the intersection of the two lists. Boston adoption 1 african 1 animals 28 arabic 1 architecture 38 art 222 asian 30 astronomy 13 baseball 94 basketball 78 berklee 285 boating 7 books 160 boston 7 boston.com 1020 boston.gov 395 boston-latin-hs 109 bpl 153 bu 101 business 191 cathedral-hs 15 children 73 church 469 climbing 23 comedy 174 comics 1 community 883 conferences 96 confernces 1 conflict-resolution 7 cycling 21 dance 156 dining 4 diving 18 dorchester 4 east-boston-hs 17 education 365 english 7 environment 12 european 4 eventbrite 452 eventful 2692 facebook 436 family 79 fashion 4 film 194 finance 2 fitness 2 food 115 football 1 french 3 games 33 german 3 government 42 green-technology 2 harvard 11 health 206 highschool 142 hiking 44 hispanic 1 history 156 hockey 35 indian 4 irish 1 islamic 15 italian 13 japanese 50 jazz 70 language 83 law 5 lectures 304 lgbt 1 library 378 martial-arts 47 massart 10 meditation 10 meetup 975 mensa 46 museum 94 music 2023 nature 91 networking 105 northeastern 167 performing-arts 222 philanthropy 1 philosophy 6 photography 53 poetry 10 politics 153 polyamory 10 portuguese 8 pub-crawl 4 recreation 201 running 59 sailing 7 science 151 seminars 10 simmons 15 social-justice 89 softball 1 south-boston-hs 1 spanish 1 spirituality 106 sports 590 statistics 2 suffolk 2 support 46 surfing 2 swimming 29 synagogue 5 technology 320 theater 90 tourism 43 tours 90 travel 3 umass 9 university 599 upcoming 212 visual-arts 72 volunteer 46 women 25 writing 17 ymca 4 yoga 27 Seattle africa 4 animals 26 aquarium 13 art 563 arts-and-crafts 14 ballet 4 basketball 31 beer 1 boating 9 books 201 business 62 business-and-technology 31 charity-and-volunteer 10 children 302 chinese 28 church 399 circus 23 cleveland-high 9 climbing 16 coffee 4 comedy 47 comics 1 community 574 conferences 65 cooking 3 dance 151 diving 8 dogs 3 education 103 environment 123 eventbrite 139 eventful 1996 facebook 216 fairs-and-festivals 13 film 136 finance 22 fitness 243 food 40 food-and-dining 26 games 114 garfield-high 12 german 13 government 145 gradeschool 17 green-technology 1 health 192 highschool 35 hiking 74 history 4 ingraham-high 1 insurance 1 italian 3 japanese 17 jazz 46 knitting 35 language 153 latin-american 30 lectures 101 lgbt 21 library 190 martial-arts 1 meetup 1107 museum 77 music 1223 native-american 20 nature 32 networking 54 nonprofit 4 nscc 1 opera 2 pacific-science-center 609 performing-arts 337 philosophy 2 photography 12 police 11 politics 31 real-estate 1 recreation 195 roosevelt-high 4 running 108 science 174 sculpture 1 seattle.gov 449 seattlepi 347 seattleu 12 seminars 23 skiing 2 spanish 19 spirituality 29 sports 151 storytelling 1 sustainability 5 swedish 73 synagogue 4 technology 98 teens 94 theater 166 tourism 11 town-hall-seattle 54 transportation 87 travel 9 trumba 296 university 390 upcoming 66 uw 366 vegan 4 visual-arts 89 volunteer 48 walk-bike-ride 3 walking 41 wallingford 159 wine 9 witches 13 women 26 writing 42 yoga 21 youth 105 Common Tags animals art basketball boating books business children church climbing comedy comics community conferences dance diving education environment eventbrite eventful facebook film finance fitness food games german government green-technology health highschool hiking history italian japanese jazz language lectures lgbt library martial-arts meetup museum music nature networking performing-arts philosophy photography politics recreation running science seminars spanish spirituality sports synagogue technology theater tourism travel university upcoming visual-arts volunteer women writing yoga Among the dynamics in play here, we can see the general and specific principle at work. For a general tag like university there are city-specific instantiations: bu and northeastern for Boston, uw and seattleu and nscc for Seattle. Likewise for the general tag highschool there are specific tags like boston-latin-hs and cathedral-hs for Boston, garfield-high and ingraham-high for Seattle. These city-specific tags are top-down in the sense that I, as curator of the hub, have assigned them and made them part of the hub’s core tag vocabulary. But they are also bottom-up in the sense that they represent discoverable sources that are providing enough event flow to warrant such treatment. These core hub vocabularies are fluid. As I move from hub to hub I’ve been keeping an eye on the common core and refactoring all the hub vocabularies as I go along. I also use these evolving hub vocabularies as templates against which to match vocabularies from other sources. Mechanism: Tag matching Some of the source services, notably Eventful and EventBrite, include per-event tags. When one of these tags matches a tag in the (evolving) core vocabulary for that hub, the elmcity service adds that tag to the event’s list of tags which it inherited from its feed. There are also tables for each foreign service that map tags used there to tags in the hub’s core vocabulary. So, for example, the Eventful tag movies_film and the EventBrite tag movies both map to the core tag film. As we saw in Portable tags, some iCalendar feeds use the CATEGORIES property of the iCalendar format to express per-event tags. Managing these tags is trickier because, well, they’re unmanaged. Until recently I was suppressing them. Now I’m experimentally allowing them to appear, but segregating them from the core vocabulary. If you check the tags for Boston or Seattle or another city you’ll see that the list divides into two sections. The first presents managed tags: the core vocabulary. The second presents unmanaged tags from iCalendar feeds, enclosed in squiggly brackets to differentiate them from the core vocabulary. Here’s the current set of unmanaged tags for Boston and Seattle: Boston {academics} 10 {adams street} 4 {air pollution control commission hearings} 1 {alumni relations} 6 {athletics} 6 {bikes} 1 {blc} 3 {boston home center} 2 {boston main streets} 1 {boston public library} 3 {brighton} 2 {central library} 84 {charlestown} 2 {city clerk} 15 {city council} 8 {college of arts & sciences} 10 {college of business administration} 1 {college of computer & information science} 13 {college of engineering} 13 {commercial} 1 {connolly} 12 {dnd} 1 {dudley literacy center} 11 {dudley} 19 {east boston} 9 {egleston square} 3 {elderly commission} 1 {election} 1 {faneuil} 2 {fields corner} 11 {group exercise} 4 {grove hall} 11 {honan- allston} 11 {hyde park} 10 {jamaica plain} 7 {licensing} 1 {lower mills} 5 {massart events} 1 {mattapan} 15 {north end} 11 {ongoing} 33 {orient heights} 5 {other} 90 {parker hill} 10 {performing/visual arts} 60 {president} 1 {public event} 139 {public health commission} 1 {roslindale} 8 {social} 6 {south boston} 18 {south end} 6 {student affairs} 2 {student development} 9 {uphams corner} 2 {washington village} 4 {west end} 13 {west roxbury} 4 Seattle {animal shelter} 23 {athletics/varsity sports/men} 3 {athletics/varsity sports/women} 4 {athletics} 6 {boards & commissions} 32 {bothell} 29 {built environments} 3 {career management} 1 {city council} 88 {community centers} 29 {community outreach} 10 {community technology} 18 {concerts} 17 {continuing education} 20 {diversity} 2 {eastside } 58 {emergency} 12 {engineering} 13 {environmental learning} 3 {exhibits} 97 {farther afield} 10 {forums} 8 {global health} 1 {health sciences} 18 {hearing examiner} 12 {hr-benefits} 1 {jackson school of international studies} 6 {libraries} 1 {meetings} 5 {north sound} 19 {office of the mayor} 2 {other} 1 {panel discussions} 2 {parks} 2 {performing/visual arts} 29 {psychology} 4 {ptsa} 6 {public outreach and engagement} 68 {public} 23 {readings} 1 {research} 1 {sales} 1 {school of art} 92 {school of business} 22 {schoolof art} 1 {seattle area} 188 {seattle fire department} 2 {seattle youth commission} 11 {south sound} 21 {special events} 16 {sports/spirit} 1 {student activities} 6 {tacoma} 3 {technical communication} 8 {the center for wooden boats – south lake union} 9 {tours} 27 {training} 13 {urbanization} 1 {vst} 1 {walk bike ride} 3 {workshops} 6 When one of these tags matches a tag in a hub’s core vocabulary I promote it — that is, I treat it as part of the managed core and it no longer shows up in squigglies. That’s a top-down approach. But there’s a complementary bottom-up approach. As I scan the unmanaged tags, both within and across hubs, it can become clear that an unmanaged tag belongs in the managed core. To accomplish that I simply use the unmanaged tag somewhere in the managed core. From then on, occurrences of the unmanaged tag are promoted into the core. A logical next step is to enable curators to edit per-hub maps so that, for example, Seattle’s {central library} and Boston’s {libraries} will be promoted to simply library. I haven’t built this mapping feature yet but it’s on the todo list. I’m still exploring the interplay between the top-down and bottom-up approaches. But it definitely feels like the right way to handle common vocabularies augmented by different (and regionally-varying) vocabularies. (This series: elmcity tagging principles.)
Danger RoomiBattle: Apple May Finally Storm The Pentagon

Marine Corps Commandant Gen. James Amos gets an iPad demo. Photo: USMC

The Pentagon is, for all intents and purposes, Apple-free zone. Its desktop computers have long run on Windows, and now its tablets and smartphones are all Android. But there’s a chance that might be about to change. The Air Force’s Air Mobility Command is considering a monster purchase of iPads — one that might give Apple an inroads into a military market that’s eluded it so far.

A recent solicitation from Air Mobility Command has the Air Force’s cargo and transport division seeking “a maximum of 18,000 iPad 2s” to serve as “Electronic Flight Bags” — essentially, tools to launch apps that digitize the flyboys’ weighty manuals. Even that purchase isn’t a slam dunk for Cupertino, since the solicitation called for iPads or “equal” devices. (Cue the indignation from Cult of Mac-types.) But if the Air Force even buys half that many iPads, it would still represent the largest tablet or smartphone purchase the military’s made thus far.

And it also could represent something of a reversal of Apple’s military fortunes. Last week, the Air Force’s Special Operations Command cancelled an order of over 2,000 iPads, apparently prompted by security concerns about Russian software on the devices for reading PDFs of flight charts that double as encryption agents. As if to twist the knife, a spokeswoman told NextGov that the command “continues” to evaluate “the procurement aspect of providing tablets to the field.” In other words, they love tablets — just not Apple’s tablet.

That’s a familiar story for Apple. While the military is just getting started playing around with smart devices, it’s balked at shelling out for Cupertino’s mobile products.

In 2010, the military’s Special Operations Command wanted a suite of apps to get militarily relevant data out to their dismounted, networked commandos, including a “multi-touch whiteboarding aka John Madden tool.” They went with Android phones to host it. The next year, when the app-mad Army started playing around with its first smartphone linked to its new data network, Android powered it.

Later in 2011, the Army bought a bunch of commercial smartphones — as in, officers went to Best Buy and purchased 60 phones — to test out how they worked with the new data network. It also debuted a new quasi-smartphone to replace an old, funky plan to make soldiers wear computers. Yet again, in both cases, Droid did.

The preference was rarely made explicit, and behind the scenes, Army officials whispered to reporters that they didn’t want to make it seem like they were dissing Apple. And Apple had a few mobile military successes, like when Marine aviators bought 32 iPads last fall to help program close air support.

But the word’s gotten around to the defense industry. At military gear showcases like the Association of the United States Army conference, developers who come up with everything from insta-translators to mapping tools run the software they want to sell the Army with Android as their operating system.

So why not Apple? As the iPad cancellation indicated, there are concerns about data security — but in fairness, the Army hasn’t yet certified Android (or Windows Mobile, for that matter) as secure, either. The answer you hear most often is cost. Apple products are more expensive, which is why most government desktops run Windows and bureaucrats thumb their way through BlackBerries. Now that the military is facing a future of tighter budgets, iPads and iPhones might be too expensive.

But the Air Mobility Command is bucking that trend. Buying 18,000 iPads will make it the most always-on part of the military by a longshot. And since the Army wants to start out using smartphones for pretty much the same function — running apps that digitize its training and field manuals — it’s possible the Air Force purchase might spur a second look at iOS from the ground-pounders.

Still, the military isn’t ready to issue camo-colored coverings to iPads just yet. An Air Mobility Command spokeswoman said the 18,000 order may not just buy iPads — Motorola’s Xoom, Samsung’s Galaxy Tab, and even Barnes & Noble’s Nook e-reader might be included in the Air Force purchase. And all those tablets have a certain competing operating system in common.

Photo: Flickr/MC4 Army


LWN.net[$] Subtle interactions in the embedded world - what bugs can teach us
Neil Brown writes: "One of the freedoms that free and open source software provides is the freedom to study the source code. However having that freedom doesn't mean it's easy to use it. When faced with a large body of code such as the Linux kernel it can be hard to know where to start, or when to move on. There is no story-line, no curriculum, no plot to guide your study. This is where a bug comes in: it can provide a story line." Click below (subscribers only) for the full story line from this week's Kernel Page.
Ars TechnicaFaster-than-light neutrino result apparently a mistake due to loose cable

Since September, scientists have been scratching their head over results that appear to show neutrinos travelling between Switzerland and Italy faster than light would. As far as anyone could tell, the team behind the results had done everything they could to eliminate errors, and had even released some preliminary data that had strengthened their results. But the results remained difficult to square with everything else we know about how the Universe operates.

But now, ScienceInsider is reporting that there was a good reason the measurements and reality weren't lining up: a loose cable was causing one of the atomic clocks used to time the neutrinos' flight to produce spurious results. If the report is confirmed, then it provides a simple explanation for the fascinating-yet-difficult-to-accept results.

Read the comments on this post


Daring Fireball‘Dell’s the Computer to Use if Taking Photos of Fat Guys Farting Is Your Thing’

I know what you’re thinking — you’re thinking this headline from The Loop’s Peter Cohen is an over-the-top bit of anti-Dell sensationalism. But it’s actually an accurate description of this promotional video from Dell.

 ★ 
Github Archiloqueworldlywisdom started watching archiloque/rest-client
rest-client's description:
Simple HTTP and REST client for Ruby, inspired by microframework syntax for specifying actions.
Dubious QualityFarewell, My Lovely
I did something three weeks ago that I never thought I would ever do.

I deleted ESPN from my browser bookmarks.

I still remember the entirely giddy thrill the first time I saw ESPN. A 24- hour SPORTS NETWORK. Are you kidding me? It was wonderful. It was wonderful for a long time, too.

At some point, though, it started a long, slow decline. ESPN stopped covering sports and started covering itself. It became endlessly self-referential. I saw a Deadspin story a few months ago that said something like "everything at ESPN masturbates everything else at ESPN," and that's a perfect description. ESPN has become the broadcast equivalent of the athlete that constantly glosses himself in the third person.

It's also entirely curious that as ESPN has gained channels, they've actually covered fewer stories. There will be two or three stories, and ESPN will absolutely pounded those stories to death.

Sometimes, it's even worse than that.

I listen to Dan Patrick's show (A+) in the morning on the radio, but when he goes to commercial, I'll switch over to ESPN radio for a few minutes and listen to Colin Cowherd.

I know, Cowherd is a blowhard and a C+ at best, but he's better than listening to commercials.

In the last five weekdays, I've turned over to Cowherd's show nine times. Every time, he's mentioned Jeremy Lin within the first 60 seconds, and usually within the first 15 seconds.

It's not just him, either. Eli likes to watch SportsCenter before school, and it was all about Lin as well. Constantly.

Now don't get me wrong. I love Jeremy Lin, and I think he is an absolutely great story, because the kid can flat-out play basketball. But it's not the only interesting story in the sports world right now.

The other problem with ESPN only covering a few stories is that their coverage in no way makes a meaningful extension to the story. There's no journalism whatsoever. It's just endless repetition with only minor variation.

Deadspin does an amusing analysis of weekly content in the 11 p.m. SportsCenter. Here's an excerpt from their most recent "Bristolmetrics":
It's a Linfestation!: Lin's 350 mentions destroyed the record of 154 mentions set by Tim Tebow in the first edition of Bristolmetrics. That shakes out to 0.93 Jeremy Lin mentions per minute. Jeremy Lin's name was uttered more times than "if" (132) or "but" (241); his last name alone was mentioned 291 times, more than "are" (229), "be" (216), or "what" (207). Lin was discussed so much that Carmelo Anthony moved into the ranks of the most-mentioned athletes despite not playing a game since Feb. 6—all 37 mentions of Anthony came in the context of discussing whether or not his return would ruin the Lin magic.

Let's express this another way: The NHL needed six weeks to log more than 50 minutes total of SportsCenter airtime; in a single week, the Knicks got 58.5 minutes all by themselves. The saddest thing I witnessed this week, amid the joy of Linsanity, was NHL analyst Barry Melrose giving over his meager share of airtime to talk about Jeremy Lin with Linda Cohn. Poor Barry.

The other problem I have with ESPN is that they seem to have only two types of on-air personalities these days: ones that are wholly indistinguishable from each other, and a second type that basically does nothing but yell. They've taken a ton of interesting sportswriters and turned them into nothing more than shouting buffoons. So I will still watch SportsCenter with Eli, but otherwise, I can find better product elsewhere.

If you're wondering if I've missed pulling up the ESPN website half a dozen times a day, the answer is "no." Yahoo Sports is far more informative in terms of actual journalism, and they've stolen some of ESPNs best writers (Pat Forde and Bruce Feldman for college football coverage, in particular). And I read Pro Football Talk, College Football Talk, and Pro Hockey Talk (all NBC-affiliated sites) every day, which are both informative as well is interesting. Filling in any gaps with old favorites like Sports Illustrated, I don't feel like I'm missing anything at all.

There's not really anything left to miss.
Planet IntertwinglyCollaborate and edit anywhere with the updated Google Docs for Android
As I was sitting on the ferry commuting to Google’s Sydney office this morning, two thoughts occurred to me. First, Australia is beautiful. If you’ve never been here, you really should visit. And second, it’s amazing how productive I can be with just my Android phone and an Internet connection. I was responding to email, reading news articles and editing documents—just like I do at the office. Only the view was better! We want to give everyone the chance to be productive no matter where they are, so today we’re releasing a new update to the Google Docs app for Android. We've brought the collaborative experience from Google Docs on the desktop to your Android device. You'll see updates in real time as others type on their computers, tablets and phones, and you can just tap the document to join in. We also updated the interface to make it easier to work with your documents on the go. For example, you can pinch to zoom and focus on a specific paragraph or see the whole document at a glance. We also added rich text formatting so you can do things like create a quick bullet list, add color to your documents, or just bold something important. Watch the new Google Docs app in action: If you want to hear about the latest Docs news or send us feedback on the new app, visit Google Docs on Google+. Gotta run—I’ve got another ferry to catch!
Planet IntertwinglyCouchbase Housewarming Party
Join us in celebrating our big move into brand new offices! We're throwing a HUGE housewarming party at our new space in Mountain View. And we have quite the entertainment lined up - trust us, you don't want to miss this event. RSVP now to get your spot at this event as we break in the new office (and more importantly, the KEGBOT). Any and all are invited to come celebrate with us! Our office is pretty damn cool. RSVP now
A Softer WorldA Softer World: 776

back
buy this print digg facebook reddit stumbleupon
next


LWN.netWednesday's security updates

CentOS has updated cvs (C6: command execution via hostile proxy server), libxml2 (C6: denial of service), and libpng (C6: code execution).

Debian has updated libmodplug (six code-execution vulnerabilities).

Fedora has updated java-1.6.0-openjdk (F15: nine CVE numbers), firefox (F15: code execution), xulrunner (F15: code execution) and thunderbird (F15: code execution).

Mandriva has updated libxml2 (denial of service) and libpng (code execution).

Oracle has updated libxml2 (EL6: denial of service) and cvs (EL6: command execution).

Red Hat has updated libxml2 (RHEL5-6: denial of service), httpd (RHEL5: multiple vulnerabilities) java-1.6.0-openjdk (RHEL5: nine CVE numbers), and cvs (RHEL5-6: code execution via hostile proxy server).

Ubuntu has updated cvs (command execution via hostile proxy server).

Dubious QualityTribes Ascend
Of note (thanks RPS):
Tribes: Ascend developers Hi-Rez Studios have crumbled to the pressure I exerted with my catchy protest chant and will be opening the beta of their online shooter on Friday Feb 24th, at exactly 5pm.

Here's the game website: Tribes Ascend.

I'm putting this up because two people I highly respect have told me, in no uncertain terms, that Tribes: Ascend is entirely badass (and they both played Tribes 2 for 200+ hours). So if you were a fan of the Tribes series, this looks to be an entirely worthy successor.
Planet Intertwingly40
My 40th birthday is coming up and I’m kind of freaked out about it. I’m no cult-of-youth guy and have a healthy fatalistic sense of the course of human life. There’s just no denying that 40 is a pretty big turning point, particularly physically. Happily I’ll be spending my actual birthday in France at a fabulously decadent restaurant, so at least I’ll be getting a good start on my gout.
Ars TechnicaAs ACTA support falters, treaty referred to European court

The prospects for quick European approval of the Anti-Counterfeiting Trade Agreement dimmed Wednesday as the European Commission, the EU's executive branch, announced plans to seek an opinion from the European Court of Justice about ACTA's constitutionality.

In a statement, Viviane Reding, the EU Commissioner for Justice, Fundamental Rights, and Citizenship, reiterated her commitment to "a freely accessible Internet" and "freedom of expression and freedom of information via the Internet." She said the EC has decided to ask the ECJ for an opinion to "clarify that the ACTA agreement and its implementation must be fully compatible with freedom of expression and freedom of the Internet."

"I believe that putting ACTA before the European Court of Justice is a needed step," she wrote in a Wednesday press release explaining the move. "This debate must be based upon facts and not upon the misinformation or rumor that has dominated social media sites and blogs in recent weeks."

"Intellectual property is Europe's main raw material, but the problem is that we currently struggle to protect it outside the European Union," Reding wrote. "This hurts our companies, destroys jobs and harms our economies. This is where ACTA will change something for all of us - as it will help protect jobs that are currently lost because counterfeited and pirated goods worth 200 billion Euros are floating around on the world markets."

The European Commission has traditionally been a strong supporter of the ACTA agreement. The executive body adopted the treaty in December, and it is due to be considered by the European Parliament later this year. Presumably, the commission hopes that a favorable ruling from the high court will ease the agreement's passage by rebutting critics' charges that ACTA threatens Internet freedom.

But the decision to seek a judicial opinion could also further delay consideration of the treaty, giving opponents more time to organize against it.

ACTA opponents have been particularly effective in Poland, where the government suspended ratification of the treaty earlier this month. As our sister site Wired has reported, online activists persuaded Polish Prime Minister Donald Tusk to participate in a 7-hour online conversation about ACTA on Twitter, Facebook, and even IRC. Last week, Tusk repudiated his earlier support for ACTA and called on other European nations to reject the treaty as well.

Read the comments on this post


Planet IntertwinglyNew PaySwarm Alpha Launched for Developers
In May of last year, we launched the first public PaySwarm system for developers. The system implemented the open standards-based, patent-and-royalty free, PaySwarm specifications that enable developers to perform Web Payments. We have learned quite a bit from that deployment, which resulted in core changes to the specification and developer website. Today, we are pleased...
Planet IntertwinglyTEST
Github Archiloquebertomartin started watching archiloque/rest-client
rest-client's description:
Simple HTTP and REST client for Ruby, inspired by microframework syntax for specifying actions.
Krebs on SecurityFeds Request DNSChanger Deadline Extension

Extradition of Accused Masterminds Moves Forward

Millions of computers infected with the stealthy and tenacious DNSChanger Trojan may be spared a planned disconnection from the Internet early next month if a New York court approves a new request by the U.S. government. Meanwhile, six men accused of managing and profiting from the huge collection of hacked PCs are expected to soon be extradited from their native Estonia to face charges in the United States.

DNSChanger modifies settings on a host PC that tell the computer how to find Web sites on the Internet, hijacking victims’ search results and preventing them from visiting security sites that might help detect and scrub the infections. The Internet servers that were used to control infected PCs were located in the United States, and in coordination with the arrest of the Estonian men in November, a New York district court ordered a private U.S. company to assume control over those servers. The government argued that the arrangement would give ISPs and companies time to identify and scrub infected PCs, systems that would otherwise be disconnected from the Internet if the control servers were shut down. The court agreed, and ordered that the surrogate control servers remain in operation until March 8.

But earlier this month, security firm Internet Identity revealed that the cleanup process was taking a lot longer than expected: The company said more than 3 million systems worldwide — 500,000 in the United States — remain infected with the Trojan, and that at least one instance of the Trojan was still running on computers at 50 percent of Fortune 500 firms and half of all U.S. government agencies. That means that if the current deadline holds, millions of PCs are likely to be cut off from the Web on March 8.

In a Feb. 17 filing with the U.S. District Court for the Southern District of New York, officials with the U.S. Justice Department, the U.S. Attorney for the Southern District of New York, and NASA asked the court to extend the March 8 deadline by more than four months to give ISPs, private companies and the government more time to clean up the mess. The government requested that the surrogate servers be allowed to stay in operation until July 9, 2012. The court has yet to rule on the request, a copy of which is available here (PDF).

Not everyone thinks extending the deadline is the best way to resolve the situation. In fact, security-minded folks seem dead-set against the idea. KrebOnSecurity conducted an unscientific poll earlier this month, asking readers whether they thought the government should give affected users more time to clean up infections from the malware, which can be unusually difficult to remove. Nearly 1,400 readers responded that forcing people to meet the current deadline was the best approach. The overwhelming opinion (~9:1) was against extending the March 8 deadline.

KrebsOnSecurity readers voted almost 9-1 against the idea of extending the Mar. 8 deadline.

In related news, the six Estonian men arrested and accused of building and profiting from the DNSChanger botnet are expected to be extradited to face computer intrusion and conspiracy charges in the United States.  According to the Baltic Business News, an Estonian court ruled last week that the country can extradite four of the six (two were already cleared for extradition). The story notes that the final decision on the extradition will be made by the Estonian government after the court’s ruling has entered into force, but sources close to the investigation say the extraditions are all but assured.

Image courtesy Eesti Päevaleht.

Among those facing certain extradition is the alleged ringleader of the group, Vladimir Tsastsin, who for many years ran a domain registration firm called EstDomains that was heavily favored by cybercriminals. In 2008, ICANN, the nonprofit organization that oversees the domain registration industry, revoked EstDomains’s contract to sell new domain names, citing Tsastsin’s prior criminal convictions for forgery, money laundering and credit card fraud.

Tsastsin and the five others are alleged to have made at least $14 million selling hijacked search traffic from infected PCs to advertisers, and by swapping ads displayed on popular sites with their own ads. The government says Tsastsin laundered the ill-gotten gains by purchasing dozens of cars and real estate properties, including a number of empty lots. The infographic above, published by Eesti Päevaleht — Estonia’s largest daily news outlet — shows some of the properties Tsastsin (bottom right) and his compatriots were alleged to have purchased with the funds earned from the DNSChanger Trojan activities.

A copy of the indictments returned against Tsastsin and others is available here (PDF).

Ars TechnicaNokia rumor roundup: new Windows Phone, Symbian models coming

Nokia is gearing up for some big smartphone announcements at Mobile World Congress next week, and not all are related to Windows Phone. In addition to a few Windows Phone handsets, the company is also teasing a camera-centric Symbian phone with one of the largest camera sensors on a mobile phone yet.

First, the Symbian outlier: the Nokia 803 will have a large imaging sensor that will be Nokia's big step up to 1080p video, according to PocketNow. With a 4-inch AMOLED screen, the phone will be an all-touchscreen successor to the Nokia N8, a phone revered for its photo prowess. Nokia's teaser commercial offers little information, but has some 1080p shots of winter scenes.

Read the rest of this article...

Read the comments on this post


Ars TechnicaFailed anti-game legislation will cost California nearly $1.8 million

Trying to pass unconstitutional restrictions on a burgeoning artistic medium is not cheap. Just ask the state of California, which will end up paying $1.8 million in attorney's fees in its failed effort to restrict violent game sales to children with a law that was overturned by the Supreme Court last year.

It was widely reported last month that the state would be paying $950,000 to cover legal costs for the Entertainment Merchants Association and Entertainment Software Association, which argued against the law. But that number didn't take into account nearly $300,000 the state had paid to industry defenders during earlier court battles, and $500,000 it spent on its own side of the legal battle, as The Sacramento Bee recently reported.

But there are no regrets from the sponsor of the bill. "When you fight the good fight for a cause you know is right and just, and it's about protecting kids, you don't ever regret that," Republican bill sponsor Leland Yee told The Bee. "I think we felt the issue was so important that it warranted the costs associated with it," former California deputy attorney general Jim Humes added.

And while the legal costs are a drop in the bucket compared to the state's massive annual budget of over $92 billion, some say legislators should have known that the law would end up being a waste of time and money. "I think it's fair to say the industry warned the state that they were just getting themselves into a big legal mess and they would end up having to pay attorney fees—and that's exactly what happened," game industry attorney Paul M. Smith told The Bee.

The good news is that states seem unlikely to waste taxpayer money on this specific issue in the future. The 7-2 Supreme Court ruling that overturned the California law was very clear in granting games full First Amendment protections, setting a precedent that other states seem unlikely to challenge directly. But that doesn't mean they won't try to find other ways to limit the impact of games they deem objectionable, as proven by an Oklahoma representative's recent efforts to add a surtax on games rated T and up.

Read the comments on this post


LWN.netKuhn: Busybox GPL enforcement concerns resolved
On the Busybox list, Bradley Kuhn (of the Software Freedom Conservancy) reports that he had a discussion with Tim Bird at the Embedded Linux Conference and was able to address most of Tim's concerns with regard to how GPL enforcement around Busybox is done. "From my point of view, my discussion with Tim settles settles the matter. Tim got some incorrect information about BusyBox enforcement efforts, and that's what led him to feel he needed to support a BusyBox replacement initially. Tim seems to be in completely reasonable [sic] about the whole thing now that he's talked directly with me about the actual GPL enforcement efforts by Conservancy for BusyBox." (Those who are just tuning in to this story can find some background in this article).
Ars TechnicaEnvironment researcher admits leaking climate docs, claims they're genuine

Last week, several documents that purportedly came from the Heartland Institute appeared on the Web, laying out the organization's financial efforts to undercut the mainstream understanding of climate science. Although the Heartland admitted that most of the materials were genuine, it claims they had been obtained via deception, and that one of the documents (the most inflammatory) was a fake. Now, a prominent environmental researcher has admitted that he impersonated a Heartland board member in order to obtain the documents, but claims they are all genuine.

Peter Gleick is the founder and current president of the Pacific Institute, where he specializes in research on the water cycle. His research can be provocative—some of it suggested that the US has already passed peak water—but has been considered important enough to get him elected to the National Academies of Science.

Read the rest of this article...

Read the comments on this post


Ars TechnicaApple confirms plans for Oregon data center, outlines green initiatives

Apple's massive North Carolina data center is apparently only the beginning when it comes to handling the deluge of content traffic flowing through Apple's servers. The company has now confirmed that it has plans to build another large data center in Prineville, Oregon, continuing the green efforts that it initiated with its east coast facility.

The news came via Oregon news station KTVZ, which was following up on rumors from December about the project as Apple spokesperson Kristin Huguet confirmed that Apple had indeed purchased the land for the data center. Huguet declined to offer any further details about Apple's plans, but KTVZ says the property's price tag was $5.6 million and the deed was signed by county commissioners on February 15. The location for the new 160-acre data center is said to be a "stone's throw" away from another large facility owned by Facebook.

Before Apple publicly acknowledged its North Carolina facility, rumors spread for years about what the company was planning to do with it. As it turned out, the NC data center ended up powering iCloud—which launched in October with iOS 5—and Apple began talking about its plans build a 171-panel solar farm to help run it. Apple's Oregon data center is expected to boast similar green efforts, but no details have been offered. Apple just laid out this week some of the other green initiatives it's taking in North Carolina, so it's likely we'll see some of those same elements carried over to the facility in the Beaver State.

Read the comments on this post


Planet IntertwinglyWool
As you may know, I participate in a project called SFShorts where we write Sci Fi in 140 characters or less. It's a lot of fun.A few weeks ago, a new member was added to our team, one Hugh Howey, who is an actual real-life science fiction author, not just a wannabe like me. This was very exciting, but I hadn't actually read any of his stuff yet. At Elizabeth's recommendation, I purchased Wool - all five parts - for my Kindle, and on my recent trip, I started reading it.Wool is a post-apocalypse dystopian novel. Folks live in a subterranean silo, and the rules have to be pretty strict to keep things working smoothly in a completely sealed environment. This gives all sorts of plot opportunities. It is, in short, a gripping book, which I read all the way through and was left wanting more MORE MORE!!!Hugh is a wonderful story teller. His characters are real people, not flat single-feature personalities, and you truly identify with folks - the good guys and the bad guys - in a way that many authors simply can't achieve.I will be buying everything Hugh writes, and pestering him to write more. You should too. This is really great stuff.By the way, for those looking for good books for kids, I should mention that the language is a little on the salty side in Wool, so use your judgement here. Better yet, read it yourself first, and figure out what your kids can handle.
Ars TechnicaFirst non-game apps show PS Vita's wider potential amidst frustration

These days, it seems, its not enough for a digital device to just play games. To keep up with the smartphones and tablet computers of the world, any game system needs to at least nod in the direction of cloud-based and social networking "apps" that are all the rage with the kiddies. Sony's PlayStation Vita has now done exactly that, launching free downloadable Netflix, Twitter and Flickr apps in conjunction with the system's official debut today (though pre-orderers have had the system for a week now).

Read the rest of this article...

Read the comments on this post


Planet IntertwinglyDaily links for 02/22/2012
John Thompson To Join Microsoft Board “Microsoft Corp. (MSFT) said Monday that John Thompson was appointed to the company’s board of directors, returning the board’s size to 10 members.” tags: microsoft Ubuntu for Android: Canonical brings Ubuntu desktop to docked smartphones “Canonical has announced a new product called Ubuntu for Android that will bring the popular Linux distribution to high-end Android smartphones. The product consists of a complete Ubuntu desktop experience that is intended to be installed on the device alongside the standard Android environment.” tags: android linux ubuntu mobile Posted from Diigo. The rest of my favorite links are here. © Robert S. Sutor for Bob Sutor, 2012. All rights reserved. Permalink | No comments Categorized under: Daily Links. Tagged with: . Twitter: bob_sutor
Ars TechnicaMicrosoft to EC: Motorola hamstringing Xbox, PC with huge patent royalties

Microsoft today filed a competition law complaint against Motorola Mobility with the European Commission, claiming that the company—and, by extension, its soon-to-be-owner Google—is charging outrageously high licensing fees for patents essential to complying with industry standards.

In a blog post titled "Google, Please Don't Kill Video on the Web," Microsoft general counsel Dave Heiner said Motorola "is attempting to block sales of Windows PCs, our Xbox game console and other products" by charging unusually high fees for patents related to the H.264 video standard. On a $1,000 laptop, Motorola wants a royalty of $22.50, Microsoft claims. Microsoft said its complaint, which is not a publicly available document, is filed against both Motorola and Google.

Read the rest of this article...

Read the comments on this post


Rezo.netNotre pays s'enfonce dans les ténèbres du Moyen Âge
« Un complot international est en cours, visant à mener à terme la destruction de mon pays. Les assaillants ont commencé en 1975, avec comme cible la culture grecque moderne, puis ils ont poursuivi la décomposition de notre histoire récente et de notre identité nationale et aujourd'hui ils essaient de nous exterminer physiquement par le chômage, la famine et la misère. Si le peuple grec ne se soulève pas pour les arrêter, le risque de disparition de la Grèce est bien réel. Je la vois arriver dans les dix prochaines années. Le seul élément qui va survivre de notre pays sera la mémoire de notre civilisation et de nos luttes pour la liberté. » Le compositeur Grec Mikis Theodorakis, ancien résistant et héros de la lutte contre le régime des colonels, aujourd'hui âgé de 87 ans, lance un appel à l'opinion internationale.
Source: ContreInfo
LWN.netMageia 2 beta 1 available
For those interested in helping to test the next Mageia distribution release: the first Mageia 2 beta is out. New features are listed on this page, they include a switch to systemd, the latest desktop environments, a switch from MySQL to MariaDB, and more.
Rezo.netLa Hongrie en marche arrière
Planet Intertwingly[2b2k] The Surprisingly Free interview
Surprisingly Free has posted its podcast interview with me, by Jerry Brito. Unsurprisingly, it’s free!
Planet IntertwinglyBig data in the cloud
Sections IaaS and private clouds Platform solutions Big data cloud platforms compared Conclusion Big data and cloud technology go hand-in-hand. Big data needs clusters of servers for processing, which clouds can readily provide. So goes the marketing message, but what does that look like in reality? Both "cloud" and "big data" have broad definitions, obscured by considerable hype. This article breaks down the landscape as simply as possible, highlighting what's practical, and what's to come. IaaS and private clouds What is often called "cloud" amounts to virtualized servers: computing resource that presents itself as a regular server, rentable per consumption. This is generally called infrastructure as a service (IaaS), and is offered by platforms such as Rackspace Cloud or Amazon EC2. You buy time on these services, and install and configure your own software, such as a Hadoop cluster or NoSQL database. Most of the solutions I described in my Big Data Market Survey can be deployed on IaaS services. Using IaaS clouds doesn't mean you must handle all deployment manually: good news for the clusters of machines big data requires. You can use orchestration frameworks, which handle the management of resources, and automated infrastructure tools, which handle server installation and configuration. RightScale offers a commercial multi-cloud management platform that mitigates some of the problems of managing servers in the cloud. Frameworks such as OpenStack and Eucalyptus aim to present a uniform interface to both private data centers and the public cloud. Attracting a strong flow of cross industry support, OpenStack currently addresses computing resource (akin to Amazon's EC2) and storage (parallels Amazon S3). The race is on to make private clouds and IaaS services more usable: over the next two years using clouds should become much more straightforward as vendors adopt the nascent standards. There'll be a uniform interface, whether you're using public or private cloud facilities, or a hybrid of the two. Particular to big data, several configuration tools already target Hadoop explicitly: among them Dell's Crowbar, which aims to make deploying and configuring clusters simple, and Apache Whirr, which is specialized for running Hadoop services and other clustered data processing systems. Today, using IaaS gives you a broad choice of cloud supplier, the option of using a private cloud, and complete control: but you'll be responsible for deploying, managing and maintaining your clusters. Microsoft SQL Server is a comprehensive information platform offering enterprise-ready technologies and tools that help businesses derive maximum value from information at the lowest TCO. SQL Server 2012 launches next year, offering a cloud-ready information platform delivering mission-critical confidence, breakthrough insight, and cloud on your terms; find out more at www.microsoft.com/sql. Platform solutions Using IaaS only brings you so far for with big data applications: they handle the creation of computing and storage resources, but don't address anything at a higher level. The set up of Hadoop and Hive or a similar solution is down to you. Beyond IaaS, several cloud services provide application layer support for big data work. Sometimes referred to as managed solutions, or platform as a service (PaaS), these services remove the need to configure or scale things such as databases or MapReduce, reducing your workload and maintenance burden. Additionally, PaaS providers can realize great efficiencies by hosting at the application level, and pass those savings on to the customer. The general PaaS market is burgeoning, with major players including VMware (Cloud Foundry) and Salesforce (Heroku, force.com). As big data and machine learning requirements percolate through the industry, these players are likely to add their own big-data-specific services. For the purposes of this article, though, I will be sticking to the vendors who already have implemented big data solutions. Today's primary providers of such big data platform services are Amazon, Google and Microsoft. You can see their offerings summarized in the table toward the end of this article. Both Amazon Web Services and Microsoft's Azure blur the lines between infrastructure as a service and platform: you can mix and match. By contrast, Google's philosophy is to skip the notion of a server altogether, and focus only on the concept of the application. Among these, only Amazon can lay claim to extensive experience with their product. Amazon Web Services Amazon has significant experience in hosting big data processing. Use of Amazon EC2 for Hadoop was a popular and natural move for many early adopters of big data, thanks to Amazon's expandable supply of compute power. Building on this, Amazon launched Elastic Map Reduce in 2009, providing a hosted, scalable Hadoop service. Applications on Amazon's platform can pick from the best of both the IaaS and PaaS worlds. General purpose EC2 servers host applications that can then access the appropriate special purpose managed solutions provided by Amazon. As well as Elastic Map Reduce, Amazon offers several other services relevant to big data, such as the Simple Queue Service for coordinating distributed computing, and a hosted relational database service. At the specialist end of big data, Amazon's High Performance Computing solutions are tuned for low-latency cluster computing, of the sort required by scientific and engineering applications. Elastic Map Reduce Elastic Map Reduce (EMR) can be programmed in the usual Hadoop ways, through Pig, Hive or other programming language, and uses Amazon's S3 storage service to get data in and out. Access to Elastic Map Reduce is through Amazon's SDKs and tools, or with GUI analytical and IDE products such as those offered by Karmasphere. In conjunction with these tools, EMR represents a strong option for experimental and analytical work. Amazon's EMR pricing makes it a much more attractive option to use EMR, rather than configure EC2 instances yourself to run Hadoop. When integrating Hadoop with applications generating structured data, using S3 as the main data source can be unwieldy. This is because, similar to Hadoop's HDFS, S3 works at the level of storing blobs of opaque data. Hadoop's answer to this is HBase, a NoSQL database that integrates with the rest of the Hadoop stack. Unfortunately, Amazon does not currently offer HBase with Elastic Map Reduce. DynamoDB Instead of HBase, Amazon provides DynamoDB, its own managed, scalable NoSQL database. As this a managed solution, it represents a better choice than running your own database on top of EC2, in terms of both performance and economy. DynamoDB data can be exported to and imported from S3, providing interoperability with EMR. Google Google's cloud platform stands out as distinct from its competitors. Rather than offering virtualization, it provides an application container with defined APIs and services. Developers do not need to concern themselves with the concept of machines: applications execute in the cloud, getting access to as much processing power as they need, within defined resource usage limits. To use Google's platform, you must work within the constraints of its APIs. However, if that fits, you can reap the benefits of the security, tuning and performance improvements inherent to the way Google develops all its services. AppEngine, Google's cloud application hosting service, offers a MapReduce facility for parallel computation over data, but this is more of a feature for use as part of complex applications rather than for analytical purposes. Instead, BigQuery and the Prediction API form the core of Google's big data offering, respectively offering analysis and machine learning facilities. Both these services are available exclusively via REST APIs, consistent with Google's vision for web-based computing. BigQuery BigQuery is an analytical database, suitable for interactive analysis over datasets of the order of 1TB. It works best on a small number of tables with a large number of rows. BigQuery offers a familiar SQL interface to its data. In that, it is comparable to Apache Hive, but the typical performance is faster, making BigQuery a good choice for exploratory data analysis. Getting data into BigQuery is a matter of directly uploading it, or importing it from Google's Cloud Storage system. This is the aspect of BigQuery with the biggest room for improvement. Whereas Amazon's S3 lets you mail in disks for import, Google doesn't currently have this facility. Streaming data into BigQuery isn't viable either, so regular imports are required for constantly updating data. Finally, as BigQuery only accepts data formatted as comma-separated value (CSV) files, you will need to use external methods to clean up the data beforehand. Rather than provide end-user interfaces itself, Google wants an ecosystem to grow around BigQuery, with vendors incorporating it into their products, in the same way Elastic Map Reduce has acquired tool integration. Currently in beta test, to which anybody can apply, BigQuery is expected to be publicly available during 2012. Prediction API Many uses of machine learning are well defined, such as classification, sentiment analysis, or recommendation generation. To meet these needs, Google offers its Prediction API product. Applications using the Prediction API work by creating and training a model hosted within Google's system. Once trained, this model can be used to make predictions, such as spam detection. Google is working on allowing these models to be shared, optionally with a fee. This will let you take advantage of previously trained models, which in many cases will save you time and expertise with training. Though promising, Google's offerings are in their early days. Further integration between its services is required, as well as time for ecosystem development to make their tools more approachable. Microsoft I have written in some detail about Microsoft's big data strategy in Microsoft's plan for Hadoop and big data. By offering its data platforms on Windows Azure in addition to Windows Server, Microsoft's aim is to make either on-premise or cloud-based deployments equally viable with its technology. Azure parallels Amazon's web service offerings in many ways, offering a mix of IaaS services with managed applications such as SQL Server. Hadoop is the central pillar of Microsoft's big data approach, surrounded by the ecosystem of its own database and business intelligence tools. For organizations already invested in the Microsoft platform, Azure will represent the smoothest route for integrating big data into the operation. Azure itself is pragmatic about language choice, supporting technologies such as Java, PHP and Node.js in addition to Microsoft's own. As with Google's BigQuery, Microsoft's Hadoop solution is currently in closed beta test, and is expected to be generally available sometime in the middle of 2012. Big data cloud platforms compared The following table summarizes the data storage and analysis capabilities of Amazon, Google and Microsoft's cloud platforms. Intentionally excluded are IaaS solutions without dedicated big data offerings.   Amazon Google Microsoft Product(s) Amazon Web Services Google Cloud Services Windows Azure Big data storage S3 Cloud Storage HDFS on Azure Working storage Elastic Block Store AppEngine (Datastore, Blobstore) Blob, table, queues NoSQL database DynamoDB1 AppEngine Datastore Table storage Relational database Relational Database Service (MySQL or Oracle) Cloud SQL (MySQL compatible) SQL Azure Application hosting EC2 AppEngine Azure Compute Map/Reduce service Elastic MapReduce (Hadoop) AppEngine (limited capacity) Hadoop on Azure2 Big data analytics Elastic MapReduce (Hadoop interface3) BigQuery2 (TB-scale, SQL interface) Hadoop on Azure (Hadoop interface3) Machine learning Via Hadoop + Mahout on EMR or EC2 Prediction API Mahout with Hadoop Streaming processing Nothing prepackaged: use custom solution on EC2 Prospective Search API 4 StreamInsight2 ("Project Austin") Data import Network, physically ship drives Network Network Data sources Public Data Sets A few sample datasets Windows Azure Marketplace Availability Public production Some services in private beta Some services in private beta Conclusion Cloud-based big data services offer considerable advantages in removing the overhead of configuring and tuning your own clusters, and in ensuring you pay only for what you use. The biggest issue is always going to be data locality, as it is slow and expensive to ship data. The most effective big data cloud solutions will be the ones where the data is also collected in the cloud. This is an incentive to investigate EC2, Azure or AppEngine as a primary application platform, and an indicator that PaaS competitors such as Cloud Foundry and Heroku will have to address big data as a priority. It is early days yet for big data in the cloud, with only Amazon offering battle-tested solutions at this point. Cloud services themselves are at an early stage, and we will see both increasing standardization and innovation over the next two years. However, the twin advantages of not having to worry about infrastructure and economies of scale mean it is well worth investigating cloud services for your big data needs, especially for an experimental or green-field project. Looking to the future, there's no doubt that big data analytical capability will form an essential component of utility computing solutions. Notes: 1 In public beta. 2 In controlled beta test. 3 Hive and Pig compatible. 4 Experimental status. Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work. Save 20% on registration with the code RADAR20 Related: Big data market survey: Hadoop solutions Microsoft's plan for Hadoop and big data Get started with Hadoop: From evaluation to your first production cluster On the performance of clouds
LWN.netNo more Flash for Firefox on Linux
Adobe has announced that its proprietary Flash plugin will be moving to a new "Pepper" API for interaction with the browser, leaving the old Netscape plugin API behind. "For Flash Player releases after 11.2, the Flash Player browser plugin for Linux will only be available via the 'Pepper' API as part of the Google Chrome browser distribution and will no longer be available as a direct download from Adobe. Adobe will continue to provide security updates to non-Pepper distributions of Flash Player 11.2 on Linux for five years from its release."
Alliance GeostrategiqueL’abécédaire du drone

Alors qu’hier était fêté (sans cotillon ni champagne) le 11ème anniversaire du premier tir d’un drone sur une cible terrestre, l’Alliance Géostratégique se donne pour objectif de revenir durant les prochaines semaines sur les différentes problématiques engendrées par l’emploi passé, présent et futur de tels outils.

En effet, que de changements depuis le tir par un drone RQ-1 Predator de ce missile AGM-114 Hellfire sur une vieille carcasse rouillée de char dormant sur la base de Nellis dans le Nevada ! Que les questions soient techniques, éthiques, stratégiques, tactiques, politiques, elles ne manquent pas et se posent et se poseront, sans doute avec encore plus d’acuité.

Pour saisir, imparfaitement et partiellement, le vaste champ de questionnement, voici un abécédaire introductif qui en présentent quelques aspects. Il peut être complété par vous et vos contributions, que nous espérons nombreuses, pourront venir creuser l’un ou l’autre des aspects (à envoyer à alliancegeostrategique@gmail.com).

A –  Applications : piloter un drone depuis un iPhone, un iPad ou un téléphone portable équipé du système d’exploitation Android, c’est possible. De nombreuses applications permettent de telles fonctionnalités, piloter mais aussi recevoir les vidéos prises. iPhone ayant même sorti un drone quadricoptère (plusieurs drones pouvant être mis en réseau).

B – Baleine : la relation est un peu capillo-tractée, quoique… L’association écologiste Sea Shepherd utilise des drones (cf. compte-rendu sur son site) pour traquer les flottes de baleiniers japonais : objectif pour le Hangar 18 Osprey trouver le bateau usine au milieu de l’océan Austral.

C – Cyber : en 2009, des logiciels à 25$ auraient permis d’exploiter une faille dans la sécurisation des liaisons de données entre les drones Predator et leurs stations au sol. Plus récemment, et bien que le mode opératoire reste flou, la saisie du drone RQ-170 Sentinel au-dessus de l’Iran, pose des questions en termes de sécurité des systèmes d’information.

D – Drunken Predator : parmi les 10 comptes Twitter retenus par le Foreign Policy pour sa série consacrée aux comptes parodiques ayant un rapport avec les affaires internationales, le compte Drunkenpredator sort du lot à côté de celui d’Angela Merkel. Plus de 6000 abonnés, pour celui qui ne souffle pas dans le ballon avant de décoller.

E – Espace aérien : des aéronefs sans pilote à bord dans l’espace aérien civil ? Beaucoup de questions techniques et réglementaires demeurent alors que de rares pays autorisent leur emploi (comme à la frontière américano-mexicaine) et que tous travaillent sur cette question (cf l’expérimentation récente de l’Italie).

F – France : Nous n’avons pas de drone mais nous avons des idées ! En employant des drones lors de la Guerre du Golfe (vous le saviez ?), les armées française étaient dans le bon wagon. Aujourd’hui avec quelques SDTI, quatre Harfang et des drones micro-tactiques, elles sont en queue de train… Quant aux débats sur leur armement ou non, vaste sujet.

G – Guerre : les drones conduisent-ils à une évolution ou révolution de l’art de la guerre ? Évolution bien sûr ! Malgré les titres médiatiques aguicheurs. Le visage de la guerre change, les outils et les pratiques ne sont plus celles d’hier, mais la nature profonde n’évolue pas : le duel des volontés demeure.

H – Histoire : les premiers drones pourraient être des ballons autrichiens sans pilote lâchant des bombes au-dessus de Venise en 1849. Ensuite, de nombreux projets virent le jour, en particulier pour les cibles aériennes commandées à distance (cf. l’utilisation toujours d’actualité dans des centres de tir par le 17è régiment d’Artillerie en France).

I – Industrie : Dassault Aviation, EADS, Rafael, Elbit,, IAI, Sagem, Schiebel, General Atomics, Northrop Grumman, etc. Les enjeux économiques (94 milliards de $ d’ici 2020 ? dont 36 rien que pour les USA ?) sont énormes pour les majeures du secteur, des challengers émergeant aussi doucement pour exacerber la concurrence.

J – Journalisme : illustration d’une utilisation civile possible, les drones pourraient être de plus en plus employés par les journalistes, citant le cas d’une manifestation filmée à Varsovie. L’article oublie de préciser que plutôt que d’être un amateur lambda, l’utilisateur du drone était plutôt un manifestant l’utilisant pour filmer les policiers anti-émeute…

M – Morale : il aurait été possible de mettre « éthique ». Quelles sont les questions éthiques et morales (reliées à des questions de légalité) causées par cette administration de la mort à distance ? Les débats sur les assassinats ciblés sont ainsi un versant important sur l’introduction de principes de proportionnalité, de « la fin justifie (ou non) les moyens », etc.

N – Navalisation : milieu complexe à maîtriser, les drones investissent petit à petit le milieu maritime. C’est le cas du Camcopter S-100 de l’autrichien Schiebel testé sur le patrouilleur Adroit de DCNS ou un MQ-8 Fire Scout perdu par l’US Navy durant des opérations au-dessus de la Libye en juin 2011.

O – Opérateurs : si il n’y a pas de pilote à bord, l’humain reste dans la boucle de l’utilisation des drones. De la gestion des compétences, de la formation, du stress enduré par des opérateurs qui administrent parfois la mort à distance avant de rentrer chez eux le soir ou autres, les questions sont nombreuses pour ces presque nouveaux militaires.

P – Présence permanente : si l’endurance est l’un des facteurs techniques de ces outils, quand est-il de ce facteur pour les populations visées ou subissant l’emploi de ces drones. C’est le cas de celles des zones tribales pakistanaises qui vivent avec cette menace permanente au-dessus de la tête, un tir pouvant arrivé à tout moment.

Q – Quantité ? La quantité est-elle une qualité en soit ?

R – Renseignement : les données fournies par les drones (vidéos, signaux radios, etc.) viennent s’ajouter à l’énorme quantité d’informations (en particulier électroniques) déjà collectée et à traiter par les services de renseignement (en particulier américains). La sur information tue t’elle l’information ?

S – Stratégie ?

T – Terrorisme ?

U – ?

V – ?

W – Waziristan : région montagneuse du Nord-est du Pakistan, le Waziristan est réputé pour être la zone abritant les état-major de mouvements affiliés ou portant le label Al-Qaida. Les frappes de drones se multiplient comme le montre The Bureau of Investigative Journalism dans son suivi minutieux de ces questions, la dernière remontant au 16 février.

Y – Yémen : après l’Afghanistan, le Pakistan ou la Somalie, le Yemen est un des principaux théâtres d’utilisation de drones pour surveiller ou frapper la mouvance AQAP (Al-Qaeda in the Arabian Peninsula). L’imam Anwar al-Awlaki en est mort le 30 septembre, faisant perdre à AQ un rédacteur en chef de qualité pour sa revue Inspire.

Z – Zéro mort : slogan « marketing » qui voit le jour durant la Guerre du Golfe de 1991 et l’emploi intensif de frappes dites de précision. L’emploi des drones est censé répondre à cette volonté (et ce besoin) de préserver les vies humaines (euphémisme qui nie l’in-niable, la mort possible…), en particulier dans son propre camp en automatisant certaines taches dangereuses.

Et maintenant, à vos claviers !

NB : « drone » est pris dans cet article dans son acceptation étroite d’aéronef sans pilote humain à bord, mais l’acceptation plus large de « mobiles robotisés » peut (et devrait) évidemment être incluse dans le débat.

AltDevBlogADayConstraint Based Design

In some cases the most destructive action one can perform on the creative psyche is to give it absolute freedom. The blank page, blue sky, and empty word document are among the most terrifying monsters in the creative world.

  1. Apply artificial Constraint
  2. Design within Constraint
  3. Remove Contraint
  4. Analyze

That sums up a little bit of advice I’m going impart as to how I defeat these monsters. I’ve used this model in various ways and below I’ll give some examples of this simple piece of advice.

One Button

The modern gamepad (or keyboard/mouse) provides a massive subset of control options. Too many in most cases. As a programmer I often just find myself going “bind” crazy because its easy to bind to a key. This leads to terrible interfaces; original Blender UI anyone ;)

As a designer I often force myself to only have one button and one stick on a gamepad, or maybe just mouse interaction. I force myself to use less buttons than I think I need. Then I will often find ways to contextualize or simplify a mechanic or control. Leading to a more elegant control solution.

This also means I can often later in a project arbitrarily say something like, “Okay we will map that Global button to Shoulder button, they aren’t used anywhere”. Which is great when in crunch or a great feature occurs late in development.

Box It

Often when doodling an idea I will just draw a box on the page. Then draw only inside that box. This only really works for visual designs but I find it works really well to focus once I have a set boundary.

Coin Toss

Have two options?

Do a blind coin toss, and then before revealing the coin if you find yourself wishing for heads or tails you know your answer.

Three Point System

When constructing a narrative it’s easy to lose sight of the overall structure or lose detail. One trick I use is what I call the Bullet Point System. The system that forces groups of three means your always forced to find that third thing but also that you often self prune. Some of my best ideas come while reaching for that third point to fit. I also find that often something sounds great but then I can’t flesh it out to three points so I discard it.

Rules

Story of Lost Boy

Pick It

Alternatively called the F it system this is for when often I’m uncertain or given too many options. If there are 6 different ways we can go and we can’t be sure which to go well then just throw hands in the air, F-it, and pick one.

Most importantly I document the choice!

I will force myself to finish the design or complete going down the path, no regrets. Then if possible when we have more time I go back to that fork in the road and re-examine the choice. Though to be frank it’s rare that you find yourself going back and re-evaulting.

Conclusion

Keep It Simple Stupid! This is all very basic advice that I was hesitant to post but then I recall watching a 30 minute cooking show on cooking Potatoes and thought well sometimes simple advice is very useful. ;)

Planet IntertwinglyData for the public good
Sections Financial good Transit data as economic fuel Transparency and civic goods Principles for data in the public good The case for open data The promise of data journalism Open aid and development Crisis data and emergency response Healthcare What lies ahead Public data is a public good Download this free report Can data save the world? Not on its own. As an age of technology-fueled transparency, open innovation and big data dawns around the world, the success of new policy won't depend on any single chief information officer, chief executive or brilliant developer. Data for the public good will be driven by a distributed community of media, nonprofits, academics and civic advocates focused on better outcomes, more informed communities and the new news, in whatever form it is delivered. Advocates, watchdogs and government officials now have new tools for data journalism and open government. Globally, there's a wave of transparency that will wash over every industry and government, from finance to healthcare to crime. In that context, open government is about much more than open data — just look at the issues that flow around the #opengov hashtag on Twitter, including the nature identity, privacy, security, procurement, culture, cloud computing, civic engagement, participatory democracy, corruption, civic entrepreneurship or transparency. If we accept the premise that Gov 2.0 is a potent combination of open government, mobile, open data, social media, collective intelligence and connectivity, the lessons of the past year suggest that a tidal wave of technology-fueled change is still building worldwide. The Economist's support for open government data remains salient today: "Public access to government figures is certain to release economic value and encourage entrepreneurship. That has already happened with weather data and with America's GPS satellite-navigation system that was opened for full commercial use a decade ago. And many firms make a good living out of searching for or repackaging patent filings." As Clive Thompson reported at Wired last year, public sector data can help fuel jobs, and "shoving more public data into the commons could kick-start billions in economic activity." In the transportation sector, for instance, transit data is open government fuel for economic growth. There is a tremendous amount of work ahead in building upon the foundations that civil society has constructed over decades. If you want a deep look at what the work of digitizing data really looks like, read Carl Malamud's interview with Slashdot on opening government data. Data for the public good, however, goes far beyond government's own actions. In many cases, it will happen despite government action — or, often, inaction — as civic developers, data scientists and clinicians pioneer better analysis, visualization and feedback loops. For every civic startup or regulation, there's a backstory that often involves a broad number of stakeholders. Governments have to commit to open up themselves but will, in many cases, need external expertise or even funding to do so. Citizens, industry and developers have to show up to use the data, demonstrating that there's not only demand, but also skill outside of government to put open data to work in service accountability, citizen utility and economic opportunity. Galvanizing the co-creation of civic services, policies or apps isn't easy, but tapping the potential of the civic surplus has attracted the attention of governments around the world. There are many challenges for that vision to pass. For one, data quality and access remain poor. Socrata's open data study identified progress, but also pointed to a clear need for improvement: Only 30% of developers surveyed said that government data was available, and of that, 50% of the data was unusable. Open data will not be a silver bullet to all of society's ills, but an increasing number of states are assembling platforms and stimulating an app economy. Results-oriented mayors like Rahm Emanuel and Mike Bloomberg are committing to opening Chicago and opening government data in New York City, respectively. Following are examples of where data for the public good is already having an impact upon the world we live in, along with some ideas about what lies ahead. Financial good Anyone looking for civic entrepreneurship will be hard pressed to find a better recent example than BrightScope. The efforts of Mike and Ryan Alfred are in line with traditional entrepreneurship: identifying an opportunity in a market that no one else has created value around, building a team to capitalize on it, and then investing years of hard work to execute on that vision. In the process, BrightScope has made government data about the financial industry more usable, searchable and open to the public. Due to the efforts of these two entrepreneurs and their California-based startup, anyone who wants to learn more about financial advisers before tapping one to manage their assets can do so online. Prior to BrightScope, the adviser data was locked up at the Securities and Exchange Commission (SEC) and the Financial Industry Regulatory Authority (FINRA). "Ryan and I knew this data was there because we were advisers," said BrightScope co-founder Mike Alfred in a 2011 interview. "We knew data had been filed, but it wasn't clear what was being done with it. We'd never seen it liberated from the government databases." While they knew the public data existed and had their idea years ago, Alfred said it didn't happen because they "weren't in the mindset of being data entrepreneurs" yet. "By going after 401(k) first, we could build the capacity to process large amounts of data," Alfred said. "We could take that data and present it on the web in a way that would be usable to the consumer." Notably, the government data that BrightScope has gathered on financial advisers goes further than a given profile page. Over time, as search engines like Google and Bing index the information, the data has become searchable in places consumers are actually looking for it. That's aligned with one of the laws for open data that Tim O'Reilly has been sharing for years: Don't make people find data. Make data find the people. As agencies adapt to new business relationships, consumers are starting to see increased access to government data. Now, more data that the nation's regulatory agencies collected on behalf of the public can be searched and understood by the public. Open data can improve lives, not least through adding more transparency into a financial sector that desperately needs more of it. This kind of data transparency will give the best financial advisers the advantage they deserve and make it much harder for your Aunt Betty to choose someone with a history of financial malpractice. The next phase of financial data for good will use big data analysis and algorithmic consumer advice tools, or "choice engines," to make better decisions. The vast majority of consumers are unlikely to ever look directly at raw datasets themselves. Instead, they'll use mobile applications, search engines and social recommendations to make smarter choices. There are already early examples of such services emerging. Billshrink, for example, lets consumers get personalized recommendations for a cheaper cell phone plan based on calling histories. Mint makes specific recommendations on how a citizen can save money based upon data analysis of the accounts added. Moreover, much of the innovation in this area is enabled by the ability of entrepreneurs and developers to go directly to data aggregation intermediaries like Yodlee or CashEdge to license the data. EMC's Big Data solution accelerates business transformation. We offer a cost-efficient and scale-out IT infrastructure that allows organizations to access broad data sources, collaborate and execute real-time analysis and drive actionable insight. Transit data as economic fuel Transit data continues to be one of the richest and most dynamic areas for co-creation of services. Around the United States and beyond, there has been a blossoming of innovation in the city transit sector, driven by the passion of citizens and fueled by the release of real-time transit data by city governments. Francisca Rojas, research director at the Harvard Kennedy School's Transparency Policy Project, has investigated the dynamics behind the disclosure of data by transit agencies in the United States, which she calls one of the most successful implementations of open government. "In just a few years, a rich community has developed around this data, with visionary champions for disclosure inside transit agencies collaborating with eager software developers to deliver multiple ways for riders to access real-time information about transit," wrote Rojas. The Massachusetts Bay Transit Authority (MBTA) learned from Portland, Oregon's, TriMet that open data is better. "This was the best thing the MBTA had done in its history," said Laurel Ruma, O'Reilly's director of talent and a long-time resident in greater Boston, in her 2010 Ignite talk on real-time transit data. The MBTA's move to make real-time data available and support it has spawned a new ecosystem of mobile applications, many of which are featured at MBTA.com. There are now 44 different consumer-facing applications for the TriMet system. Chicago, Washington and New York City also have a growing ecosystem of applications. As more sensors go online in smarter cities, tracking the movements of traffic patterns will enable public administrators to optimize routes, schedules and capacity, driving efficiency and a better allocation of resources. Transparency and civic goods As John Wonderlich, policy director at the Sunlight Foundation, observed last year, access to legislative data brings citizens closer to their representatives. "When developers and programmers have better access to the data of Congress, they can better build the databases and tools that let the rest of us connect with the legislature." That's the promise of the Sunlight Foundation's work, in general: Technology-fueled transparency will help fight corruption, fraud and reveal the influence behind policies. That work is guided by data, generated, scraped and aggregated from government and regulatory bodies. The Sunlight Foundation has been focused on opening up Congress through technology since the organization was founded. Some of its efforts culminated recently with the publication of a live XML feed for the House floor and a transparency portal for House legislative documents. There are other horizons for transparency through open government data, which broadly refers to public sector records that have been made available to citizens. For a canonical resource on what makes such releases truly "open," consult the "8 Principles of Open Government Data." For instance, while gerrymandering has been part of American civic life since the birth of the republic, one of the best policy innovations of 2011 may offer hope for improving the redistricting process. DistrictBuilder, an open-source tool created by the Public Mapping Project, allows anyone to easily create legal districts. "During the last year, thousands of members of the public have participated in online redistricting and have created hundreds of valid public plans," said Micah Altman, senior research scientist at Harvard University Institute for Quantitative Social Science, via an email last year. "In substantial part, this is due to the project's effort and software. This year represents a huge increase in participation compared to previous rounds of redistricting — for example, the number of plans produced and shared by members of the public this year is roughly 100 times the number of plans submitted by the public in the last round of redistricting 10 years ago," Altman said. "Furthermore, the extensive news coverage has helped make a whole new set of people aware of the issue and has re framed it as a problem that citizens can actively participate in to solve, rather than simply complain about." Principles for data in the public good As a result of digital technology, our collective public memory can now be shared and expanded upon daily. In a recent lecture on public data for public good at Code for America, Michal Migurski of Stamen Design made the point that part of the global financial crisis came through a crisis in public knowledge, citing "The Destruction of Economic Facts," by Hernando de Soto. To arrive at virtuous feedback loops that amplify the signals that citizens, regulators, executives and elected leaders inundated with information need to make better decisions, data providers and infomediaries will need to embrace key principles, as Migurski's lecture outlined. First, "data drives demand," wrote Tim O'Reilly, who attended the lecture and distilled Migurski's insights. "When Stamen launched crimespotting.org, it made people aware that the data existed. It was there, but until they put visualization front and center, it might as well not have been." Second, "public demand drives better data," wrote O'Reilly. "Crimespotting led Oakland to improve their data publishing practices. The stability of the data and publishing on the web made it possible to have this data addressable with public links. There's an 'official version,' and that version is public, rather than hidden." Third, "version control adds dimension to data," wrote O'Reilly. "Part of what matters so much when open source, the web, and open data meet government is that practices that developers take for granted become part of the way the public gets access to data. Rather than static snapshots, there's a sense that you can expect to move through time with the data." The case for open data Accountability and transparency are important civic goods, but adopting open data requires grounded arguments for a city chief financial officer to support these initiatives. When it comes to making a business case for open data, John Tolva, the chief technology officer for Chicago, identified four areas that support the investment in open government: Trust — "Open data can build or rebuild trust in the people we serve," Tolva said. "That pays dividends over time." Accountability of the work force — "We've built a performance dashboard with KPIs [key performance indicators] that track where the city directly touches a resident." Business building — "Weather apps, transit apps ... that's the easy stuff," he said. "Companies built on reading vital signs of the human body could be reading the vital signs of the city." Urban analytics — "Brett [Goldstein] established probability curves for violent crime. Now we're trying to do that elsewhere, uncovering cost savings, intervention points, and efficiencies." New York City is also using data internally. The city is doing things like applying predictive analytics to building code violations and housing data to try to understand where potential fire risks might exist. "The thing that's really exciting to me, better than internal data, of course, is open data," said New York City chief digital officer Rachel Sterne during her talk at Strata New York 2011. "This, I think, is where we really start to reach the potential of New York City becoming a platform like some of the bigger commercial platforms and open data platforms. How can New York City, with the enormous amount of data and resources we have, think of itself the same way Facebook has an API ecosystem or Twitter does? This can enable us to produce a more user-centric experience of government. It democratizes the exchange of information and services. If someone wants to do a better job than we are in communicating something, it's all out there. It empowers citizens to collaboratively create solutions. It's not just the consumption but the co-production of government services and democracy." The promise of data journalism The ascendance of data journalism in media and government will continue to gather force in the years ahead. Journalists and citizens are confronted by unprecedented amounts of data and an expanded number of news sources, including a social web populated by our friends, family and colleagues. Newsrooms, the traditional hosts for information gathering and dissemination, are now part of a flattened environment for news. Developments often break first on social networks, and that information is then curated by a combination of professionals and amateurs. News is then analyzed and synthesized into contextualized journalism. Data is being scraped by journalists, generated from citizen reporting, or gleaned from massive information dumps — such as with the Guardian's formidable data journalism, as detailed in a recent ebook. ScraperWiki, a favorite tool of civic coders at Code for America and elsewhere, enables anyone to collect, store and publish public data. As we grapple with the consumption challenges presented by this deluge of data, new publishing platforms are also empowering us to gather, refine, analyze and share data ourselves, turning it into information. There are a growing number of data journalism efforts around the world, from New York Times interactive features to the award-winning investigative work of ProPublica. Here are just a few promising examples: Spending Stories, from the Open Knowledge Foundation, is designed to add context to news stories based upon government data by connecting stories to the data used. Poderopedia is trying to bring more transparency to Chile, using data visualizations that draw upon a database of editorial and crowdsourced data. The State Decoded is working to make the law more user-friendly. Public Laboratory is a tool kit and online community for grassroots data gathering and research that builds upon the success of Grassroots Mapping. Internews and its local partner Nai Mediawatch launched a new website that shows incidents of violence against journalists in Afghanistan. Open aid and development The World Bank has been taking unprecedented steps to make its data more open and usable to everyone. The data.worldbank.org website that launched in September 2010 was designed to make the bank's open data easier to use. In the months since, more than 100 applications have been built using the data. "Up until very recently, there was almost no way to figure out where a development project was," said Aleem Walji, practice manager for innovation and technology at the World Bank Institute, in an interview last year. "That was true for all donors, including us. You could go into a data bank, find a project ID, download a 100-page document, and somewhere it might mention it. To look at it all on a country level was impossible. That's exactly the kind of organization-centric search that's possible now with extracted information on a map, mashed up with indicators. All of sudden, donors and recipients can both look at relationships." Open data efforts are not limited to development. More data-driven transparency in aid spending is also going online. Last year, the United States Agency for International Development (USAID) launched a public engagement effort to raise awareness about the devastating famine in the Horn of Africa. The FWD campaign includes a combination of open data, mapping and citizen engagement. "Frankly, it's the first foray the agency is taking into open government, open data, and citizen engagement online," said Haley Van Dyck, director of digital strategy at USAID, in an interview last year. "We recognize there is a lot more to do on this front, but are happy to start moving the ball forward. This campaign is different than anything USAID has done in the past. It is based on informing, engaging, and connecting with the American people to partner with us on these dire but solvable problems. We want to change not only the way USAID communicates with the American public, but also the way we share information." USAID built and embedded interactive maps on the FWD site. The agency created the maps with open source mapping tools and published the datasets it used to make these maps on data.gov. All are available to the public and media to download and embed as well. The combination of publishing maps and the open data that drives them simultaneously online is significantly evolved for any government agency, and it serves as a worthy bar for other efforts in the future to meet. USAID accomplished this by migrating its data to an open, machine-readable format. "In the past, we released our data in inaccessible formats — mostly PDFs — that are often unable to be used effectively," said Van Dyck. "USAID is one of the premiere data collectors in the international development space. We want to start making that data open, making that data sharable, and using that data to tell stories about the crisis and the work we are doing on the ground in an interactive way." Crisis data and emergency response Unprecedented levels of connectivity now exist around the world. According to a 2011 survey from the Pew Internet and Life Project, more than 50% of American adults use social networks, 35% of American adults have smartphones, and 78% of American adults are connected to the Internet. When combined, those factors mean that we now see earthquake tweets spread faster than the seismic waves themselves. Networked publics can now share the effects of disasters in real time, providing officials with unprecedented insight into what's happening. Citizens act as sensors in the midst of the storm, creating an ad hoc system of networked accountability through data. The growth of an Internet of Things is an important evolution. What we saw during Hurricane Irene in 2011 was the increasing importance of an Internet of people, where citizens act as sensors during an emergency. Emergency management practitioners and first responders have woken up to the potential of using social data for enhanced situational awareness and resource allocation. An historic emergency social data summit in Washington in 2010 highlighted how relevant this area has become. And last year's hearing in the United States Senate on the role of social media in emergency management was "a turning point in Gov 2.0," said Brian Humphrey of the Los Angeles Fire Department. The Red Cross has been at the forefront of using social data in a time of need. That's not entirely by choice, given that news of disasters has consistently broken first on Twitter. The challenge is for the men and women entrusted with coordinating response to identify signals in the noise. First responders and crisis managers are using a growing suite of tools for gathering information and sharing crucial messages internally and with the public. Structured social data and geospatial mapping suggest one direction where these tools are evolving in the field. A web application from ESRI deployed during historic floods in Australia demonstrated how crowdsourced social intelligence provided by Ushahidi can enable emergency social data to be integrated into crisis response in a meaningful way. The Australian flooding web app includes the ability to toggle layers from OpenStreetMap, satellite imagery, and topography, and then filter by time or report type. By adding structured social data, the web app provides geospatial information system (GIS) operators with valuable situational awareness that goes beyond standard reporting, including the locations of property damage, roads affected, hazards, evacuations and power outages. Long before the floods or the Red Cross joined Twitter, however, Brian Humphrey of the Los Angeles Fire Department (LAFD) was already online, listening. "The biggest gap directly involves response agencies and the Red Cross," said Humphrey, who currently serves as the LAFD's public affairs officer. "Through social media, we're trying to narrow that gap between response and recovery to offer real-time relief." After the devastating 2010 earthquake in Haiti, the evolution of volunteers working collaboratively online also offered a glimpse into the potential of citizen-generated data. Crisis Commons has acted as a sort of "geeks without borders." Around the world, developers, GIS engineers, online media professionals and volunteers collaborated on information technology projects to support disaster relief for post-earthquake Haiti, mapping streets on OpenStreetMap and collecting crisis data on Ushahidi. Healthcare What happens when patients find out how good their doctors really are? That was the question that Harvard Medical School professor Dr. Atul Gawande asked in the New Yorker, nearly a decade ago. The narrative he told in that essay makes the history of quality improvement in medicine compelling, connecting it to the creation of a data registry at the Cystic Fibrosis Foundation in the 1950s. As Gawande detailed, that data was privately held. After it became open, life expectancy for cystic fibrosis patients tripled. In 2012, the new hope is in big data, where techniques for finding meaning in the huge amounts of unstructured data generated by healthcare diagnostics offer immense promise. The trouble, say medical experts, is that data availability and quality remain significant pain points that are holding back existing programs. There are, literally, bright spots that suggest what's possible. Dr. Gawande's 2011 essay, which considered whether "hotspotting" using health data could help lower medical costs by giving the neediest patients better care, offered another perspective on the issue. Early outcomes made the approach look compelling. As Dr. Gawande detailed, when a Medicare demonstration program offered medical institutions payments that financed the coordination of care for its most chronically expensive beneficiaries, hospital stays and trips to the emergency rooms dropped more than 15% over the course of three years. A test program adopting a similar approach in Atlantic City saw a 25% drop in costs. Through sharing data and knowledge, and then creating a system to convert ideas into practice, clinicians in the ImproveCareNow network were able to improve the remission rate for Crohn's disease from 49% to 67% without the introduction of new drugs. In Britain, researchers found that the outcomes for adult cardiac patients improved after the publication of information on death rates. With the release of meaningful new open government data about performance and outcomes from the British national healthcare system, similar improvements may be on the way. "I do believe we are at the beginning of a revolutionary moment in health care, when patients and clinicians collect and share data, working together to create more effective health care systems," said Susannah Fox, associate director for digital strategy at the Pew Internet and Life Project, in an interview in January. Fox's research has documented the social life of health information, the concept of peer-to-peer healthcare, and the role of the Internet among people living with chronic disease. In the past few years, entrepreneurs, developers and government agencies have been collaboratively exploring the power of open data to improve health. In the United States, the open data story in healthcare is evolving quickly, from new mobile apps that lead to better health decisions to data spurring changes in care at the U.S. Department of Veterans Affairs. Since he entered public service, Todd Park, the first chief technology officer of the U.S. Department of Health and Human Services (HHS), has focused on unleashing the power of open data to improve health. If you aren't familiar with this story, read the Atlantic's feature article that explores Park's efforts to revolutionize the healthcare industry through better use of data. Park has focused on releasing data at Health.Data.Gov. In a speech to a Hacks and Hackers meetup in New York City in 2011, Park emphasized that HHS wasn't just releasing new data: "[We're] also making existing data truly accessible or usable," he said, taking "stuff that's in a book or on a website and turning it into machine-readable data or an API." Park said it's still quite early in the project and that the work isn't just about data — it's about how and where it's used. "Data by itself isn't useful. You don't go and download data and slather data on yourself and get healed," he said. "Data is useful when it's integrated with other stuff that does useful jobs for doctors, patients and consumers." What lies ahead There are four trends that warrant special attention as we look to the future of data for public good: civic network effects, hybridized data models, personal data ownership and smart disclosure. Civic network effects Community is a key ingredient in successful open government data initiatives. It's not enough to simply release data and hope that venture capitalists and developers magically become aware of the opportunity to put it to work. Marketing open government data is what repeatedly brought federal Chief Technology Officer Aneesh Chopra and Park out to Silicon Valley, New York City and other business and tech hubs. Despite the addition of topical communities to Data.gov, conferences and new media efforts, government's attempts to act as an "impatient convener" can only go so far. Civic developer and startup communities are creating a new distributed ecosystem that will help create that community, from BuzzData to Socrata to new efforts like Max Ogden's DataCouch. Smart disclosure There are enormous economic and civic good opportunities in the "smart disclosure" of personal data, whereby a private company or government institution provides a person with access to his or her own data in open formats. Smart disclosure is defined by Cass Sunstein, Administrator of the White House Office for Information and Regulatory Affairs, as a process that "refers to the timely release of complex information and data in standardized, machine-readable formats in ways that enable consumers to make informed decisions." For instance, the quarterly financial statements of the top public companies in the world are now available online through the Securities and Exchange Commission. Why does it matter? The interactions of citizens with companies or government entities generate a huge amount of economically valuable data. If consumers and regulators had access to that data, they could tap it to make better choices about everything from finance to healthcare to real estate, much in the same way that web applications like Hipmunk and Zillow let consumers make more informed decisions. Personal data assets When a trend makes it to the World Economic Forum (WEF) in Davos, it's generally evidence that the trend is gathering steam. A report titled "Personal Data Ownership: The Emergence of a New Asset Class" suggests that 2012 will be the year when citizens start thinking more about data ownership, whether that data is generated by private companies or the public sector. "Increasing the control that individuals have over the manner in which their personal data is collected, managed and shared will spur a host of new services and applications," wrote the paper's authors. "As some put it, personal data will be the new 'oil' — a valuable resource of the 21st century. It will emerge as a new asset class touching all aspects of society." The idea of data as a currency is still in its infancy, as Strata Conference chair Edd Dumbill has emphasized. The Locker Project, which provides people with the ability to move their own data around, is one of many approaches. The growth of the Quantified Self movement and online communities like PatientsLikeMe and 23andMe validates the strength of the movement. In the U.S. federal government, the Blue Button initiative, which enables veterans to download personal health data, has now spread to all federal employees and earned adoption at Aetna and Kaiser Permanente. In early 2012, a Green Button was launched to unleash energy data in the same way. Venture capitalist Fred Wilson called the Green Button an "OAuth for energy data." Wilson wrote: "It is a simple standard that the utilities can implement on one side and web/mobile developers can implement on the other side. And the result is a ton of information sharing about energy consumption and, in all likelihood, energy savings that result from more informed consumers." Hybridized public-private data Free or low-cost online tools are empowering citizens to do more than donate money or blood: Now, they can donate, time, expertise or even act as sensors. In the United States, we saw a leading edge of this phenomenon in the Gulf of Mexico, where Oil Reporter, an open source oil spill reporting app, provided a prototype for data collection via smartphone. In Japan, an analogous effort called Safecast grew and matured in the wake of the nuclear disaster that resulted from a massive earthquake and subsequent tsunami in 2011. Open source software and citizens acting as sensors have steadily been integrated into journalism over the past few years, most dramatically in the videos and pictures uploaded after the 2009 Iran election and during 2011's Arab Spring. Citizen science looks like the next frontier. Safecast is combining open data collected by citizen science with academic, NGO and open government data (where available), and then making it widely available. It's similar to other projects, where public data and experimental data are percolating. Public data is a public good Despite the myriad challenges presented by legitimate concerns about privacy, security, intellectual property and liability, the promise of more informed citizens is significant. McKinsey's 2011 report dubbed big data as the next frontier for innovation, with billions of dollars of economic value yet to be created. When that innovation is applied on behalf of the public good, whether it's in city planning, transit, healthcare, government accountability or situational awareness, those effects will be extended. We're entering the feedback economy, where dynamic feedback loops between customers and corporations, partners and providers, citizens and governments, or regulators and companies can both drive efficiencies and leaner, smarter governments. The exabyte age will bring with it the twin challenges of information overload and overconsumption, both of which will require organizations of all sizes to use the emerging toolboxes for filtering, analysis and action. To create public good from public goods — the public sector data that governments collect, the private sector data that is being collected and the social data that we generate ourselves — we will need to collectively forge new compacts that honor existing laws and visionary agreements that enable the new data science to put the data to work. Photo: NYTimes: 365/360 - 1984 (in color) by blprnt_van, on Flickr Related: Download a free copy of "Data for the public good" (EPUB, Mobi and PDF)
Rezo.netSOS capitalisme, le film
C'est vrai que l'heure est grave. La tempête socialiste se précise. Et menace d'emporter avec elle 50 ans d'acquis fiscaux. « La gauche et son socialisme sauvage, dérégulé veulent détruire le pacte républicain. Ils veulent la mort des conseils d'administration » s'est emporté le président candidat. « Je ne laisserai pas la gauche faire des riches les éternels boucs-émissaires de la République ».
On a donc décidé de faire un film. Un film engagé. Pour briser cette oligarchie de gauchistes efféminés qui règne sur la culture. « Les grosses tapettes » comme les appelle malicieusement Gontrand. (...)
Rezo.netLes droits fondamentaux, toujours pas d'actualité en prison
Fouilles corporelles abusives, secret médical pas respecté, travail toujours précaire... Le contrôleur général des lieux de privation de liberté rend son rapport ce mercredi.
Source: Libération
Rezo.netDette publique, la conjuration des bonnes idées
En s'accordant, lors du sommet européen du 9 décembre 2011, sur un nouveau pacte budgétaire intergouvernemental, les chefs d'Etat européens ne se sont pas seulement entendus sur une condamnation des peuples de l'Union aux fers et aux chaînes de la rigueur perpétuelle, ils ont aussi pactisé sur le renoncement à deux idées qui faisaient leur chemin : faire payer les banques, comme l'avait défendu l'Allemagne pour traiter du cas de la Grèce, et encourager la Banque centrale européenne (BCE) à racheter les titres de dette des pays attaqués, comme le souhaitait la France. Donnant-donnant : il fut convenu de ne plus embêter son voisin avec une idée qui le dérangeait. Sans doute aussi qu'en fermant à clé les issues de secours, les uns et les autres pensaient apaiser les flammes de l'incendie. Les issues se rouvriront certainement sous le souffle de l'explosion. (...)
Rezo.netSyria has made a curious transition from US ally to violator of human rights
In the war on terror America was happy to send suspects to Syria. Now the US cries torture
Source: The Guardian
Rezo.netNorth Korea's dynastic succession
When North Korea's leader, Kim Jong-il, died, there was widespread concern about the consequences, especially in the West. But his son seems to have succeeded smoothly: the country will not collapse, implode or explode. The succession appears to be safe, and may last a long time
Schneier on SecurityJohn Nash's 1955 Letter to the NSA

Fascinating.

Planet IntertwinglyJohn Nash's 1955 Letter to the NSA
Fascinating.
Planet IntertwinglyWIP: Widgets and Gadgets
This is the fifth in my series of Web Integration Patterns. Check out the intro at this URL http://rollerweblogger.org/roller/entry/web_integration_patterns Synopsis Allow other web sites and applications to integrate your site into their web pages by providing an embeddable user interface, commonly known as a Gadget or Widget, which allows users to view and interact with your site in the context of other sites. Motivations By embedding Widgets in your site, you can make your site more useful and informative to your users. Users can access relevant information from other sites, in the context of your web site. By allow other sites to embed your Widgets, you can give your site and the services that it offers wider reach. Your users can access and interact with your services, in the context of other sites. Related Patterns Dashboards and Portals Social Network Plugins Delegated Pickers A Widget or Gadget is a piece of an application, a visual component or a chunk of user-interface that can be added to some other application. On the web, Widgets are implemented using HTML and JavaScript, or sometimes Flash. Here we'll discuss simple Widgets and leave discussion or more advanced usage of Widgets in Dashboards, Portals and Pickers for the more advanced patterns we'll discuss later. Easy to "consume" Widgets are easy, especially for widget "consumers" i.e. those who wish to add an existing Widget to their site. Adding a widget to your site or web application can be as easy as adding a couple of lines of HTML and JavaScript to the pages of your site. You can see some examples of Widgets in the two screen shots I've included here, below is the Wayin polling Widget, which shows my latest Wayins: And below is the Flickr Widget, which shows a random selection of my photos from Flickr. You can see these two Widgets in action on the front page of my blog. And, by the way, the Like, Tweet and +1 buttons on the right of this entry (when viewed on my blog site) are also Widgets. Relatively easy to create Developing an entirely new Widget more complicated than using one that already exists. Creating a new Widget typically requires writing HTML and JavaScript code and employing some interesting tricks. I'm not going to go into detail as there are already so many good resources on the web for learning about Widget crafting. For example, Alex Marandon wrote a pretty comprehensive blog post on creating Widgets covering basic issues like how to ensure that your widget code does not interfere with the page hosting the Widget, how to dynamically load resources and how to bypass browsers’ single-origin policy using JSON-P. Widget and Gadget standards For simple Widgets you don't really need any standards other than HTML and JavaScript. If you want to target specific web site Dashboards or Portals, like NetVibes, or enterprise web applications like Jive or Atlassian's software development tools, then you'll need to create your Widgets with specific APIs in mind. For example, NetVibes provides it's own Widget API, while Jive and Atlassian support the Open Social Gadget APIs. I'll talk more about these issues when I cover the related patterns Dashboards, Portals and Social Network Plugins. Wrapping up... That's it for Widgets and Gadgets. Next up: Feed-based Integration.
Planet IntertwinglyVideo: Nightline's report on life inside an iPad factory
Bill Weir dons a bunny suit and takes a camera into Foxconn's Shenzhen facilities Click for video With Apple's (AAPL) permission, Foxconn for the first time allowed a reporter and his camera crew into its famous Shenzhen, China, factory complex. The 17-minute report aired Tuesday night. Apart from some details about the production processes (it takes 141 steps, mostly done by hand, to assemble an iPhone) there's little news here that wasn't in the reporter's notebook Nightline co-host Bill Weir filed on Monday. But the segment is worth watching if only for his exclusive video of the endless production lines, the huge cafeterias, the crowded dorm rooms, the suicide nets, the stampede of job seekers and a glimpse of life in the villages where they come from. Weir also accompanied the Fair Labor Association as it made an Apple-sponsored audit of the facility. "I expect them to put on a show for us," FLA president Auret van Heerden told Weir. And indeed, AppleInsider Wednesday related the claims of a local labor group that Foxconn hid its underage employees before the auditors arrived. The YouTube link is pasted above. The video is also available at ABC Nightline's website here. Filed under: Apple 2.0
jwzComputer Game Bot Turing Test

Carlos Bueno wrote:

Before I talk about my own troubles, let me tell you about another book, "Computer Game Bot Turing Test". It's one of over 100,000 "books" "written" by a Markov chain running over random Wikipedia articles, bundled up and sold online for a ridiculous price. The publisher, Betascript, is notorious for this kind of thing.

It gets better. There are whole species of other bots that infest the Amazon Marketplace, pretending to have used copies of books, fighting epic price wars no one ever sees. So with "Turing Test" we have a delightful futuristic absurdity: a computer program, pretending to be human, hawking a book about computers pretending to be human, while other computer programs pretend to have used copies of it. A book that was never actually written, much less printed and read.

The internet has everything.

Last year I published my children's book about computer science, Lauren Ipsum. I set a price of $14.95 for the paperback edition and sales have been pretty good. Then last week I noticed a marketplace bot offering to sell it for $55.63. "Silly bots", I thought to myself.

Then another piled on, and then an overseas dropshipper, and then another bot. Pretty soon they were offering my book below the retail price.

The punchline is that Amazon itself is a bot. Noticing all of this activity, it decided to put the book on sale! 28% off. I can't wait to find out what that does to my margin.

My reaction to this algorithmic whipsawing has settled down to a kind of helpless bemusement. The plot of my book is about how understanding computers is the first step to taking control of your life in the 21st century. Now I don't know what to believe.

It's possible that the optimal price of Lauren Ipsum is, in fact, ten dollars and seventy-six cents and I should just relax and trust the tattooed hipster who wrote Amazon's pricing algorithm. After all, I have no choice.

But I can't help but think about that old gambler's proverb: "If you can't spot the sucker, it's you."

Previously, previously, previously.

Planet IntertwinglyYour Next Desktop Could be a Phone
Henri Sivonen: This getting interesting: Using an Android phone as an Ubuntu desktop when docked Definitely Want. Especially love the idea of sending and receiving texts from my desktop.  Would prefer a dock the size of a mac mini with a hard drive, USB and ethernet ports.
Sam RubyYour Next Desktop Could be a Phone
Henri Sivonen: This getting interesting: Using an Android phone as an Ubuntu desktop when docked Definitely Want. Especially love the idea of sending and receiving texts from my desktop.  Would prefer a dock the size of a mac mini with a hard drive, USB and ethernet ports.
Rezo.netViolation étatique et domination postcoloniale : le « modèle » algérien
L'exemple algérien peut nous éclairer sur les mécanismes qui conduisent à la reproduction d'une idéologie qui devrait avoir passé l'arme à gauche il y a une quarantaine d'années. Qu'il s'agisse du rôle des militaires dans la construction de l'État, de sa forme particulière de violence ou de la justification de son maintien en dépit de ses échecs, l'histoire d'un pays qui fête cette année ses cinquante ans d'indépendance est riche d'enseignements. (...)
Source: Article11
Planet IntertwinglyApple and Proview trade blows in a Shanghai courtroom
In a fractious four-hour hearing, both sides were admonished by the judge Proview's i-Pad The world's most valuable company and a troubled Chinese electronics manufacturer that's about to be delisted from the Hong Kong stock exchange unless it can come up with some cash squared off in a Shanghai courtroom Wednesday. At stake: the trademark for Apple's (AAPL) iPad, the most successful new electronics gadget since, well, the iPhone. Apple claims it bought worldwide rights for the trademark three years ago for $55,000. Proview, which briefly partnered with National Semiconductor to market the i-Pad, a low-cost knock-off of Apple's iMac, claims Apple slipped up and neglected to buy the rights from Shenzhen Proview, the company's Chinese subsidiary. "Apple has no right to sell iPads under that name," the lawyer for Shenzhen Proview argued, according to an AP report. Apple's iPad 2 "Proview has no product, no markets, no customers and no suppliers. It has nothing," the lawyer representing Apple countered, according to the Washington Post. "Apple has huge sales in China. Its fans line up to buy Apple products. The ban, if executed, would not only hurt Apple sales but it would also hurt China's national interest." Proview's lawyer would have none of it: "Whether people will go hungry because you can't sell iPads in China is not the issue," he said. "The court must rule according to the law. Do you absolutely have to sell the product? Can't you sell it using a different name?" More than 100 reporters packed the courtroom to watch the fractious exchange. After admonishing both sides to respect the rules of the court, the judge adjourned the hearing. His decision may rest on the details of the 2009 transaction in which a British firm, secretly working for Apple, bought what it believed were all rights to the iPad trademark from Proview International Holdings' Taiwanese subsidiary. Proview Shenzhen claims that the Chinese rights were not part of that deal. A Hong Court ruled in Apple's favor last summer, finding that Shenzhen Proview was trying to "take advantage" of the situation. In December, Proview won a judgement against Apple in a provincial Chinese court, resulting in iPads being removed from some retailers' shelves. A hearing on Apple's appeal is scheduled for Feb. 29. Proview has said it is open to a settlement, but it also sent a legal team to the U.S. to demand $2 billion in damages. "It's all about the money," reads an editorial in the English-language China Daily. "People are anxious to see whether Proview (Shenzhen), which has fallen on hard times, can force Apple to pay up for sloppy legal work establishing its trademark." Apple seems in no mood to settle. In a letter to Proview's chairman delivered Monday, it claimed to have evidence that he personally authorized the sale of all rights, Taiwanese and Chinese. The letter threatened to sue him for defamation, which in China can be punishable by death. Filed under: Apple 2.0
Danger RoomMexican Mayhem Fuels U.S. ‘Bodyguard’ Boom

Photo: gynti_46/Flickr

On the surface, the drug war across the border boils down to a conflict between Mexico’s military and rival groups of cartels. This is true, but it leaves out Mexico’s other conflict — one fought against civilians through kidnapping, extortion and assassination. Little wonder that those who can afford it are now fueling a boom in professional bodyguards and guns-for-hire, many of them based in the United States.

And the boom is not just limited to jobs in Mexico — it’s happening on both sides of the border. It involves private security firms employed by both Mexican and U.S. citizens traveling from one country to the other, to border regions, or fueled by Mexican citizens relocating to the U.S. to escape the violence. The jobs given to the bodyguards involve protecting their clients against violent cartel threats and abductions, and helping negotiate kidnapping cases.

There are also serious risks. Some companies won’t work in Mexico proper because of the danger, while others do so — quietly.

R. Kent Morrison, president of Texas security firm BlackStone Group, hates the term bodyguard. For the heavily-built ex-Navy commando and Gulf War veteran, the term suggests a “big hulking gorilla with dark sunglasses and the trench coat.” Tucked away in a small office building in the wooded suburbs of west Austin, BlackStone is among a number of companies near the border competing in the more exclusive field of “executive protection,” security industry lingo for a more elite (and expensive) brand of hired muscle.

“Historically in the U.S., security has been the stereotypical, polyester-clad, eight-dollar-an-hour security guard, and that more than anything is just a way to reduce liability insurance costs,” Morrison says. “What we do is focus on providing security to usually high-net-worth individuals who actually have a need for security and aren’t provided that by some government entity.”

But when discussing Mexico, Morrison is cautious. He says BlackStone is seeing growth from Mexican nationals, mostly business executives traveling north, but only gives a rough estimate and doesn’t give out the names of individual customers. “I’ll tell you in the last five years the requests that we’ve had for those types of services from individuals and businessmen traveling from south of the border has probably doubled,” he said.

He says having a security detail in Mexico is now a literal “status symbol” for the country’s elite. And as that elite relocates to states like Texas because of violence or travels to do business, “they want to duplicate the services that they’ve grown accustomed to down south.”

This caution to discuss specifics reflects the shadowy and hazardous nature of Mexico’s private security business. Clayton International, an “executive protection” and counter-kidnapping subsidiary of longtime Iraq mercenary group Triple Canopy, and reported to work extensively in Mexico, said it would not answer questions from Danger Room due the “sensitivity of the subject.” Philip Klein, president of Houston-area Klein Investigations and Consulting, said his company said his company has seen a 120 percent increase over the past two years, from 55 “sorties” to Mexico in 2009 to 121 last year. Klein expects the “unrest to continue down there, unless the government can get control,” and therefore more business for his company. But Klein is reluctant to discuss details.

One reason for the reluctance, according to a Triple Canopy employee speaking on background to Danger Room, is sheer risk. “The thing about working in Mexico is despite what movies or T.V. or popular perception might be is that nobody down there is armed on an executive protection detail. The Mexicans will not allow it — period,” the employee said. And not only that, if the cartels do attempt an assassination, they will “kill the protective detail too just to make sure all the loose ends are tied up.”

Mexico does indeed prohibit foreign nationals from carrying weapons. Instead, companies have subcontracted out the gun-slinging to Mexican freelancers and local firms. Rather than an armed U.S. mercenary team, a typical security detail includes two unarmed “detail leaders” from the U.S. in charge of four armed Mexican guards, hired from local firms or police officers moonlighting for extra pay. The U.S. agents call the shots and pick the travel routes, while the Mexican guards provide the muscle and firepower.

But the reliance U.S. mercs have on their armed Mexican subcontractors comes with another set of risks: the subcontractors can be easily out-bidded. Morrison said that “on a Tuesday, this group may be a completely solid group and Thursday they’ve been corrupted by the cartels.”

But whether the demand in border states like Texas is being fueled by actual threats of kidnapping or just the fear it could happen, however, is hard to ascertain. But Mexican citizens are facing a different level of risk. In a nod to the reality of the sometimes blurry distinctions between legitimate business in Mexico and organized crime, Morrison added that clients have told him: “‘A competitor made a play for my company and I have to come in and make sure they don’t strengthen that play by snatching my kid.’”


Rezo.netPhotovoltaïque : les raisons de la débâcle
À l'image du fabricant français Photowatt, l'industrie photovoltaïque allemande connaît une période difficile. Les dépôts de bilan se multiplient. En cause : la concurrence chinoise, une demande en recul, mais également une mauvaise gestion suite à l'euphorie boursière qu'a connue le solaire. Cette crise va coûter cher aux travailleurs, dans une branche où les bas salaires sont encore légion. (...)
Source: Basta !
Rezo.netCes pressings qui nous intoxiquent au perchloroéthylène
Le perchloroéthylène, solvant toxique classé cancérigène probable pour l'homme, est interdit aux États-unis et au Danemark. En France, 90 % des pressings l'utilisent encore. Outre le cancer, cet agent provoque des nausées, des troubles respiratoires et nerveux, et est impliqué dans la maladie de Parkinson. Soutenues par des scientifiques, des victimes du « perchlo » réclament son interdiction. Et menacent de porter plainte contre l'État. (...)
Source: Basta !
Planet IntertwinglyWHATWG Weekly: Unicode for the platform?
In less than a year we reached another arbitrary milestone. HTML is another thousand revisions further, now over 7000 (not quite 9000). This is the WHATWG Weekly. Over on public-script-coord@w3.org, the mailing list used by TC39 (responsible for JavaScript) and the WebApps WG to coordinate development of JavaScript, IDL, and APIs, Brendan Eich launched a mega thread on full Unicode for ES6. The entire platform is currently build around 16-bit code units, which are not quite sufficient to encompass all code points. Some code points therefore require two code units, but string manipulation, length information, etc. is all in code units, making it difficult to deal with code points that require two (in practice nobody seems to bother much). The idea is to introduce some kind of switch which when used would let you deal with code points exclusively, rather than code units. HTML did not change much last week as its editor was playing in the snow. The DOM meanwhile now has mutation observers defined, the replacement for mutation events. Adam Klein did all the heavy lifting and yours truly cleaned it up a bit. An introduction to DOM events has been added as well.
Planet IntertwinglyHTML5 adds new translate attribute
A translate attribute was recently added to HTML5. At the three MultilingualWeb workshops we have run over the past two years, the idea of this kind of ‘translate flag’ has constantly excited strong interest from localizers, content creators, and from folks working with language technology. How it works Typically authors or automated script environments will put the attribute in the markup of a page. You may also find that, in industrial translation scenarios, localizers may add attributes during the translation preparation stage, as a way of avoiding the multiplicative effects of dealing with mistranslations in a large number of languages. There is no effect on the rendered page (although you could, of course, style it if you found a good reason for doing so). The attribute will typically be used by workflow tools when the time comes to translate the text – be it by the careful craft of human translators, or by quick gist-translation APIs and services in the cloud. The attribute can appear on any element, and it takes just two values: yes or no. If the value is no, translation tools should protect the text of the element from translation. The translation tool in question could be an automated translation engine, like those used in the online services offered by Google and Microsoft. Or it could be a human translator’s ‘workbench’ tool, which would prevent the translator inadvertently changing the text. Setting this translate flag on an element applies the value to all contained elements and to all attribute values of those elements. You don’t have to use translate="yes" for this to work. If a page has no translate attribute, a translation system or translator should assume that all the text is to be translated. The yes value is likely to see little use, though it could be very useful if you need to override a translate flag on a parent element and indicate some bits of text that should be translated. You may want to translate the natural language text in examples of source code, for example, but leave the code untranslated. Why it is needed You come across a need for this quite frequently. There is an example in the HTML5 spec about the Bee Game. Here is a similar, but real example from my days at Xerox, where the documentation being translated referred to a machine with text on the hardware that wasn’t translated.

Click the Resume button on the Status Display or the CONTINUE button on the printer panel.

Here are a couple more (real) examples of content that could benefit from the translate attribute. The first is from a book, quoting a title of a work.

The question in the title How Far Can You Go? applies to both the undermining of traditional religious belief by radical theology and the undermining of literary convention by the device of "breaking frame"...

The next example is from a page about French bread – the French for bread is ‘pain‘.

Welcome to french pain on Facebook. Join now to write reviews and connect with french pain. Help your friends discover great places to visit by recommending french pain.

Adding the translate attribute to your page can help readers better understand your content when they run it through automatic translation systems, and can save a significant amount of cost and hassle for translation vendors with large throughput in many languages. What about Google Translate and Microsoft Translator? Both Google and Microsoft online translation services already provided the ability to prevent translation of content by adding markup to your content, although they did it in (multiple) different ways. Hopefully, the new attribute will help significantly by providing a standard approach. Both Google and Microsoft currently support class="notranslate", but replacing a class attribute value with an attribute that is a formal part of the language makes this feature much more reliable, especially in wider contexts. For example, a translation prep tool would be able to rely on the meaning of the HTML5 translate attribute always being what is expected. Also it becomes easier to port the concept to other scenarios, such as other translation APIs or localization standards such as XLIFF. As it happens, the online service of Microsoft (who actually proposed a translate flag for HTML5 some time ago) already supported translate="no". This, of course, was a proprietary tag until now, and Google didn’t support it. However, just yesterday morning I received word, by coincidence, that Webkit/Chromium has just added support for the translate attribute, and yesterday afternoon Google added support for translate="no" to its online translation service. See the results of some tests I put together this morning. (Neither yet supports the translate="yes" override.) In these proprietary systems, however, there are a good number of other non-standard ways to express similar ideas, even just sticking with Google and Microsoft. Microsoft apparently supports style="notranslate". This is not one of the options Google lists for their online service, but on the other hand they have things that are not available via Microsoft’s service. For example, if you have an entire page that should not be translated, you can add inside the head element of your page and Google won’t translate any of the content on that page. (However they also support .) This shouldn’t be Google specific, and a single way of doing this, ie. translate="no" on the html tag, is far cleaner. It’s also not made clear, by the way, when dealing with either translation service, how to make sub-elements translatable inside an element where translate has been set to no – which may sometimes be needed. As already mentioned, the new HTML5 translate attribute provides a simple and standard feature of HTML that can replace and simplify all these different approaches, and will help authors develop content that will work with other systems too. Can’t we just use the lang attribute? It was inevitable that someone would suggest this during the discussions around how to implement a translate flag, however overloading language tags is not the solution. For example, a language tag can indicate which text is to be spellchecked against a particular dictionary. This has nothing to do with whether that text is to be translated or not. They are different concepts. In a document that has lang="en" in the html header, if you set lang="notranslate" lower down the page, that text will now not be spellchecked, since the language is no longer English. (Nor for the matter will styling work, voice browsers pronounce correctly, etc.) Going beyond the translate attribute The W3C’s ITS (International Tag Set) Recommendation proposes the use of a translate flag such as the attribute just added to HTML5, but also goes beyond that in describing a way to assign translate flag values to particular elements or combinations of markup throughout a document or set of documents. For example, you could say, if it makes sense for your content, that by default, all p elements with a particular class name should have the translate flag set to no for a specific set of documents. Microsoft offers something along these lines already, although it is much less powerful than the ITS approach. If you use anywhere on the page (or as part of a widget snippet) it ensures that any of the CSS classes listed following “notranslateclasses” should behave the same as the “notranslate” class. Microsoft’s translation engine also doesn’t translate content within code elements. Note, however, that you don’t seem to have any choice about this – there don’t seem to be instructions about how to override this if you do want your code element content translated. By the way, there are plans afoot to set up a new MultilingualWeb-LT Working Group at the W3C in conjunction with a European Commission project to further develop ideas around the ITS spec, and create reference implementations. They will be looking, amongst many other things, at ways of integrating the new translate attribute into localization industry workflows and standards. Keep an eye out for it.
Planet IntertwinglyFour short links: 22 February 2012
Hashbangs (Dan Webb) -- why those terrible #! URLs are a bad idea. Looks like they're going away with pushState coming to browsers. As Dan says, "URLs are forever". Let's get them right. I'm fascinated by how URLs are changing meaning and use over time. DNA Sequencing on a USB Stick -- this has been going the rounds, but I think there's a time coming when scientific data generation can be crowdsourced. I care about a particular type of fish, but it hasn't been sequenced. Can I catch one, sequence it, upload the sequence, and get insight into the animal by automated detection of similar genes from other animals? Let those who care do the boring work, let scientists work on the analysis. The US Recording Industry is Stealing From Me (Bruce Simpson) -- automated content detection at YouTube has created an industry of parasites who claim copyright infringement and then receive royalties from the ads shown on the allegedly infringing videos. Ubuntu on Android -- carry a desktop in your pocket? Tempting. It's for manufacturers, not something you install on existing handsets, which I'm sure will create tension with the open source world at Ubuntu's heart. Then again, creating tension with the open source world at Ubuntu's heart does seem to be Canonical's core competency ....
Planet IntertwinglyHow To Text Mine Open Access Documents
Note: I've placed all the code and files associated with this post in a repository on GitHub. If you'd like to fork the project and add a script which processes documents through your favourite entity extractor (storing the results in a new directory), I'd be happy to receive pull requests. Fetching the documents First of all, find a set of open access documents in a standard XML format. Articles deposited in PubMed Central (PMC) are ideal, as they are converted from publisher-specific DTDs to one of the standard NLM Journal Article DTDs during deposition. PMC also has an OAI interface, which makes it straightforward to find and retrieve articles. To find the name of a set of articles, use the OAI "ListSets" command to fetch all the sets into a local CSV file. Have a look through that file and find the set you're interested in - in this case I'm using "elsevierwt": Elsevier's "Sponsored Documents", for which a fee has been paid on publication to make the articles open access; the license allows text mining for non-commercial purposes*. Use that set name with the OAI "ListIdentifiers" command to fetch the identifiers for all documents in that set into a local CSV file. This script checks that each article is also in the "pmc-open" set, which denotes the Open Access subset of PubMed Central. For each identifier, use the OAI "GetRecord" command to fetch the document XML into a local folder. The document identifier can be base64-encoded into the filename, so it's easy to identify later. Converting the documents Convert all the XML files to the most up-to-date NLM Journal Article DTD, using the XSL transformation provided by the NLM for this purpose. In this case, I'm converting from v2 to v3 of the NLM Journal Article Archiving and Interchange format; once JATS becomes the official standard hopefully the same tools will be provided for conversion. Finally, convert the body of the article to simple HTML, using another XSL transformation. All the inline elements will become "span" elements, all the block-level elements will become "div" elements**. Text mining Now the articles are ready for text mining. Choose an entity extraction tool or web service and run each article through it. I'm using the EBI's Whatizit here, which has a SOAP web service that understands plain text and returns XML. If you're lucky, you'll have a simple HTTP POST web service that understands HTML and returns JSON. Store the results, and extract the data you need. So far, I've extracted disease and protein names from these articles using Whatizit; the easiest way to find the names for the Whatizit processing pipelines is to View Source and look at the options in Whatizit's HTML entry form. I'm still pondering what to do with the extracted entities: they could be added to this GitHub repository, in which case they'll be retrievable using the base64-encoded document identifier, but maybe they'd be more accessible in a structured database somewhere with a SPARQL interface...? * This particular license is quite vague, full of restrictions, and doesn't mention what you can do with derivative works - such as the results of text mining. You might want to choose a set of articles from PLoS or BioMed Central instead, which are clearly licenced with Creative Commons CC-BY licences. ** Each element retains its attributes, and a "class" attribute is added for styling if you ever want to display this HTML.
DedefensaPlus la Grèce coule, plus la Grèce est “sauvée”, – et vice-versa
Un nouveau “bailout
Ars TechnicaApache devs release version 2.4, first major update in six years

The Apache Software Foundation has announced the availability of Apache 2.4, a major update of the popular open source HTTP server. The arrival of the new version, which is the first major release of Apache in six years, coincides with the software's 17th anniversary.

The Apache project emerged in 1995 around a fork of a Web server that was originally developed at the National Center for Supercomputing Applications (NCSA). Apache became the Web's number one HTTP server is currently used by 400 million websites around the world, powering roughly 60 percent of all active domains.

The new version of Apache introduces a number of new features and technical improvements that will help it retain its standing. The developers say that version 2.4 is significantly more efficient than its predecessor, offering better performance and lower resource consumption.

"This release delivers a host of evolutionary enhancements throughout the server that our users, administrators, and developers will welcome," Apache server vice president Eric Covener wrote in a statement. "We've added many new modules in this release, as well as broadened the capability and flexibility of existing features".

One improvement that is particularly worth noting is that the multiprocessing module system (MPM) has been improved so that the desired module can be selected at runtime. Various MPMs implement different behaviors for how the HTTP server spreads its workload across threads and processes.

Previously, the desired module had to be selected during the configuration step of the build process. In version 2.4, it's now possible to select multiple MPMs during configuration and specify the one that should be used at runtime. This will offer more flexibility in Apache deployments.

Although Apache is highly popular and deeply entrenched, it is facing some fresh competition from nginx, an open source Web server that has seen dramatic growth in adoption over the past year. The latest statistics show that nginx has become the second most popular server, surpassing Microsoft's IIS.

Adopters generally cite superior performance as the reason why they replace Apache with nginx. According to some benchmarks (PDF) demonstrated by Apache Software Foundation president Jim Jagielski, Apache 2.4 offers competitive performance.

For more details about the release, you can refer to the official launch announcement. An overview of the new features can be found in the Apache 2.4 documentation.

Read the comments on this post


Planet IntertwinglyDaniel Steinberg thinks that maybe credit is the wrong word
Daniel adds his own thoughts to my musings on why people fail to attribute: Maybe credit is the wrong word for me. Maybe what I want is an appreciation of those whose work we enjoy and build on. Linked by James Duncan Davidson.
Planet IntertwinglyMatt Haughey’s experience travelling the world with an unlocked iPhone 4S
Matt’s conclusion: international unlock rocks. I’m glad to see he’s had a great experience. I’m considering following in his footsteps after much debate and waiting about on my current contract to expire. Linked by James Duncan Davidson.
Planet IntertwinglyExpanding the Cloud – The Amazon Simple Workflow Service
Today AWS launched an exciting new service for developers: the Amazon Simple Workflow Service. Amazon SWF is an orchestration service for building scalable distributed applications. Often an application consists of several different tasks to be performed in particular sequence driven by a set of dynamic conditions. Amazon SWF makes it very easy for developers to architect and implement these tasks, run them in the cloud or on premise and coordinate their flow. Amazon SWF manages the execution flow such that the tasks are load balanced across the registered workers, that inter-task dependencies are respected, that concurrency is handled appropriately and that child workflows are executed. A growing number of applications are relying on asynchronous and distributed processing, with scalability of the application as the primary motivation. By designing autonomous distributed components, developers get the flexibility to deploy and scale out parts of the application independently as load increases. The asynchronous and distributed model has the benefits of loose coupling and selective scalability, but it also creates new challenges. Application developers must coordinate multiple distributed components to get the desired results. They must deal with the increased latency and unreliability inherent in remote communication. Components may take extended periods of time to complete tasks, requests may fail and errors originating from remote systems must be handled. Today, to accomplish this, developers are forced to write complicated infrastructure that typically involves message queues and databases along with complex logic to synchronize them. All this ‘plumbing’ is extraneous to business logic and makes the application code unnecessarily complicated and hard to maintain. Amazon SWF enables applications to be built by orchestrating tasks coordinated by a decider process. Tasks represent logical units of work and are performed by application components that can take any form, including executable code, scripts, web service calls, and human actions. Developers have full control over implementing and orchestrating tasks, without worrying about underlying complexities such as tracking their progress and keeping their state. Developers implement workers to perform tasks. They run their workers either on cloud infrastructure, such as Amazon EC2, or on-premise. Tasks can be long-running, may fail, may timeout and may complete with varying throughputs and latencies. Amazon SWF stores tasks for workers, assigns them when workers are ready, tracks their progress, and keeps their latest state, including details on their completion. To orchestrate tasks, developers write programs that get the latest state of tasks from Amazon SWF and use it to initiate subsequent tasks in an ongoing manner. Amazon SWF maintains an application’s execution state durably so that the application can be resilient to failures in individual application components. An important feature of Amazon SWF is the auditability; Amazon SWF gives visibility into the execution of each step in the application. The Management Console and APIs let you monitor all running executions of the application. The customer can zoom in on any execution to see the status of each task and its input and output data. To facilitate troubleshooting and historical analysis, Amazon SWF retains the history of executions for any number of days that the customer cab specify, up to a maximum of 90 days. Amazon SWF provides a collection of very powerful building blocks that also can be used to build higher-level execution engines. Some of our early customers used Amazon SWF to implement their domain specific languages (DL) for specialized business process execution, This is an area where I think the availability of Amazon SWF will drive a lot of innovation. As part of the AWS SDK, the AWS Flow Framework helps developers create Amazon SWF based application quickly and easily. The Java version of the SDK includes really cool integration at the langue level, making it easy for developers to automatically transform java code into tasks, create the right dependencies, and manage the execution of the workflow. This brings the power that some languages with built-in distribution and concurrency like Erlang offer to Java. For more insight into workflow execution, task coordination, task routing, task distribution, exception handling, child workflows, timers, signals, markers and much more see the Amazon SWF detail page. More information about the SDK see the developers guide. As always The AWS developer blog has additional details.
QC RSSDiscrete Happiness




Ads by Project Wonderful! Your ad could be here, right now.

Oh my gosh you guys, QC reader Glenn Case and his friend Rachael made a COMPLETELY ADORABLE version of the wake-up song from yesterday's comic! You can listen to it here, or here if that first link goes down). I am pleased beyond description. I am also working on a rather...different interpretation of the song, which I may post tomorrow if I finish it in time.

In other news, this auction is completely ridiculous.

Krebs on SecurityHow Not to Buy Tax Software

Scott Henry scoured the Web for a good deal on buying tax preparation software. His search ended at Blvdsoftware.com, which advertised a great price and an instant download. But when it came time to install the software, Henry began to have misgivings about the purchase, and reached out KrebsOnSecurity for a gut-check on whether trusting the software with his tax information was a wise move.

Five days after Henry purchased the product, blvdsoftware.com vanished from the Internet.

Several red flags should have stopped him from making the purchase. Blvdsoftware.com claimed it had been in business since 2005, but a check of the site’s WHOIS registration records showed it was created in late October 2011. The site said that Blvdsoftware was a company in Beverly Hills, Calif., but the California Secretary of State had no record of the firm, and Google Maps knew nothing of the business at its stated address.

Henry said that in years past, he’d always bought a CD version of the software. But this year, he opted for digital download.

“I was going to download from Amazon — they sell a download-only version — and then I saw the cheaper site and went with them,” he said in an email. He installed the program, but said he didn’t enter any of his sensitive data. For one thing, he never received a license key from Blvdsoftware, and the program he installed didn’t request one. Now he’s wondering if the program was — at the very least pirated — and at worst — bundled with software designed to surreptitiously snoop on his computer.

The errant buy was doubly insulting because Henry bought the software using a prepaid debit card, and now finds himself unable to dispute the charge.

Buying software from random sites or companies you know nothing about and haven’t researched is a bad idea all around. But fail to do due diligence on a bargain site that sells tax return software and you could be handing your identity and computer over to cyber thieves.

If you’re in the market for tax software downloads, save yourself the worry and hassle, and stick to known and trusted outlets online. Search for any of the titles listed at the cached version of Blvdsoftware’s site and you will probably discover that after the first page of results the vendors start to look pretty sketchy. Also, avoid using debit cards for online purchases.

If your income is $57,000 or less, you can file your taxes online for free using IRS’ Free File software, available at no charge here. And remember that the IRS does not initiate contact with taxpayers via email. If, however, you do receive a snail mail notice from the IRS about more than one tax return being filed in your name, or that you were paid by an employer you don’t know, someone may be trying to fraudulently file a tax return on your behalf. See this page from the Federal Trade Commission for more information on tax related identity theft.

AltDevBlogADayComparing Floating Point Numbers, 2012 Edition

We’ve finally reached the point in this series that I’ve been waiting for. In this post I am going to share the most crucial piece of floating-point math knowledge that I have. Here it is:

[Floating-point] math is hard.

You just won’t believe how vastly, hugely, mind-bogglingly hard it is. I mean, you may think it’s difficult to calculate when trains from Chicago and Los Angeles will collide, but that’s just peanuts to floating-point math.

Seriously. Each time I think that I’ve wrapped my head around the subtleties and implications of floating-point math I find that I’m wrong and that there is some extra confounding factor that I had failed to consider. So, the lesson to remember is that floating-point math is always more complex than you think it is. Keep that in mind through the rest of the post where we talk about the promised topic of comparing floats, and understand that this post gives some suggestions on techniques, but no silver bullets.

Previously on this channel…

This is the fifth chapter in what is currently a four chapter series. The previous posts include:

Comparing for equality

Floating point math is not exact. Simple values like 0.1 cannot be precisely represented using binary floating point numbers, and the limited precision of floating point numbers means that slight changes in the order of operations or the precision of intermediates can change the result. That means that comparing two floats to see if they are equal is usually not what you want. GCC even has a warning for this: “warning: comparing floating point with == or != is unsafe”.

Here’s one example of the inexactness that can creep in:

    float f = 0.1f;
    float sum;
    sum = 0;
 
    for (int i = 0; i < 10; ++i)
        sum += f;
    float product = f * 10;
    printf("sum = %1.15f, mul = %1.15f, mul2 = %1.15f\n",
            sum, product, f * 10);

This code shows two ancient and mystical techniques for calculating ‘one’. I call these techniques “iterative-adding” and “multiplying”. And, just to show how messy this stuff is I do the multiplication twice. That’s three separate calculations of the same thing. Naturally we get three different results, and only one of them is the unity we were seeking:

sum = 1.000000119209290, mul = 1.000000000000000, mul2 = 1.000000014901161

Disclaimer: the results you get will depend on your compiler and your compiler settings, which actually helps make the point.

So what happened, and which one is correct?

What do you mean ‘correct’?

Before we can continue I need to make clear the difference between 0.1, float(0.1), and double(0.1). In C/C++ 0.1 and double(0.1) are the same thing, but when I say “0.1” in text I mean the exact base-10 number, whereas float(0.1) and double(0.1) are rounded versions of 0.1. And, to be clear, float(0.1) and double(0.1) don’t have the same value, because float(0.1) has fewer binary digits, and therefore has more error. Here are the values for 0.1, float(0.1), and double(0.1):

Number Value
0.1 0.1 (duh)
float(0.1) 0.100000001490116119384765625
double(0.1) 0.1000000000000000055511151231257827021181583404541015625

With that settled, let’s look at the results of the code above:

  1. sum = 1.000000119209290: this calculation starts with a rounded value and then adds it ten times with potential rounding at each add, so there is lots of room for error to creep in. The final result is not 1.0, and it is not 10 * float(0.1). However it is the next representable float above 1.0, so it is very close.
  2. mul = 1.000000000000000: this calculation starts with a rounded value and then multiplies by ten, so there are fewer opportunities for error to creep in. It turns out that the conversion from 0.1 to float(0.1) rounds up, but the multiplication by ten happens to, in this case, round down, and sometimes two rounds make a right. So we get the right answer for the wrong reasons. Or maybe it’s the wrong answer, since it isn’t actually ten times float(0.1)
  3. mul2 = 1.000000014901161: this calculation starts with a rounded value and then does a double-precision multiply by ten, thus avoiding any subsequent rounding error. So we get a different right answer – the exact value of 10 * float(0.1) (which can be stored in a double but not in a float).

So, answer one is incorrect, but it is as close to 1.0 as a float can get without being there. Answer two is arguably correct, but so is answer three.

Now what?

Now we have a couple of different answers (I’m going to ignore the double precision answer), so what do we do? What if we are looking for results that are equal to one, but we also want to count any that are plausibly equal to one – results that are “close enough”.

Epsilon comparisons

If comparing floats for equality is a bad idea then how about checking whether their difference is within some error bounds or epsilon value, like this:

bool isEqual = fabs(f1 – f2) <= epsilon;

With this calculation we can express the concept of two floats being close enough that we want to consider them to be equal. But what value should we use for epsilon?

Given our experimentation above we might be tempted to use the error in our sum, which was about 1.19e7f. In fact, there’s even a define in float.h with that exact value, and it’s called FLT_EPSILON.

Clearly that’s it. The header file gods have spoken and FLT_EPSILON is the one true epsilon!

Except that that is rubbish. For numbers between 1.0 and 2.0 FLT_EPSILON represents the difference between adjacent floats. For numbers smaller than 1.0 an epsilon of FLT_EPSILON quickly becomes too large, and with small enough numbers FLT_EPSILON may be bigger than the numbers you are comparing!

For numbers larger than 2.0 the gap between floats grows larger and if you compare floats using FLT_EPSILON then you are just doing a more-expensive and less-obvious equality check. For numbers above 16777216 the appropriate epsilon to use for floats is actually greater than one, and a comparison using FLT_EPSILON just makes you look foolish. We don’t want that.

Relative epsilon comparisons

The idea of a relative epsilon comparison is to find the difference between the two numbers, and see how big it is compared  to their magnitudes. In order to get consistent results you should always compare the difference to the larger of the two numbers. In English:

To compare f1 and f2 calculate diff = fabs(f1-f2). If diff is smaller than n% of max(abs(f1),abs(f2)) then f1 and f2 can be considered equal.

In code:

bool AlmostEqualRelative(float A, float B, float maxRelDiff)
{
    // Calculate the difference.
    float diff = fabs(A - B);
    A = fabs(A);
    B = fabs(B);
    // Find the largest
    float largest = (B > A) ? B : A;
 
    if (diff <= largest * maxRelDiff)
        return true;
    return false;
}

This function is not bad. It works. Mostly. I’ll talk about the limitations later, but first I want to get to the point of this article – the technique that I first suggested many years ago.

When doing a relative comparison of floats it works pretty well to set maxRelDiff to FLT_EPSILON, or some small multiple of FLT_EPSILON. Anything smaller than that and it risks being equivalent to no epsilon. You can certainly make it larger, if greater error is expected, but don’t go crazy. However selecting the correct value for maxRelativeError is a bit tweaky and non-obvious and the lack of a direct relationship to the floating point format being used makes me sad.

ULP, he said nervously

We already know that adjacent floats have integer representations that are adjacent. This means that if we subtract the integer representations of two numbers then the difference tells us how far apart the numbers are in float space. That brings us to:

Dawson’s obvious-in-hindsight theorem:

If the integer representations of two same-sign floats are subtracted then the absolute value of the result is equal to one plus the number of representable floats between them.

In other words, if you subtract the integer representations and get one, then the two floats are as close as they can be without being equal. If you get two then they are still really close, with just one float between them. The difference between the integer representations tells us how many Units in the Last Place the numbers differ by. This is usually shortened to ULP, as in “these two floats differ by two ULPs.”

So let’s try that concept:

/* See
http://randomascii.wordpress.com/2012/01/11/tricks-with-the-floating-point-format/
for the potential portability problems with the union and bit-fields below.
*/
union Float_t
{
    Float_t(float f1 = 0.0f) : f(f1) {}
	// Portable sign-extraction
	bool Sign() const { return (i >> 31) != 0; }
 
    int32_t i;
    float f;
    struct
    {   // Bitfields for exploration. Do not use in production code.
        uint32_t mantissa : 23;
        uint32_t exponent : 8;
        uint32_t sign : 1;
    } parts;
};
 
bool AlmostEqualUlps(float A, float B, int maxUlpsDiff)
{
    Float_t uA(A);
    Float_t uB(B);
 
    // Different signs means they do not match.
    if (uA.Sign() != uB.Sign())
    {
        // Check for equality to make sure +0==-0
        if (A == B)
            return true;
        return false;
    }
 
    // Find the difference in ULPs.
    int ulpsDiff = abs(uA.i - uB.i);
    if (ulpsDiff <= maxUlpsDiff)
        return true;
 
    return false;
}

This is tricky and perhaps profound.

The check for different signs is necessary for several reasons. Subtracting the signed-magnitude representation of floats using twos-complement math isn’t particularly meaningful, and the subtraction would produce a 33-bit result and overflow. Even if we deal with these technical issues it turns out that an ULPs based comparison of floats with different signs doesn’t even make sense.

After the special cases are dealt with we simply subtract the integer representations, get the absolute value, and now we know how different the numbers are. The ‘ulpsDiff’ value gives us the number of floats between the two numbers (plus one) which is a wonderfully intuitive way of dealing with floating-point error.

One ULPs difference good (adjacent floats).

One million ULPs difference bad (kinda different).

Comparing numbers with ULPs is really just a way of doing relative comparisons. It has different characteristics at the extremes, but in the range of normal numbers it is quite well behaved. The concept is sufficiently ‘normal’ that boost has a function for calculating the difference in ULPs between two numbers.

A one ULP difference is the smallest possible difference between two numbers. One ULP between two floats is far larger than one ULP between two doubles, but the nomenclature remains terse and convenient. I like it.

ULP versus FLT_EPSILON

It turns out checking for adjacent floats using the ULPs based comparison is quite similar to using AlmostEqualRelative with epsilon set to FLT_EPSILON. For numbers that are slightly above a power of two the results are generally the same. For numbers that are slightly below a power of two the FLT_EPSILON technique is twice as lenient. In other words, if we compare 4.0 to 4.0 plus two ULPs then a one ULPs comparison and a FLT_EPSILON relative comparison will both say they are equal. However if you compare 4.0 to 4.0 minus two ULPs then a one ULPs comparison will say they are not equal (of course) but a FLT_EPSILON relative comparison will say that they are equal.

This makes sense. Adding two ULPs to 4.0 changes its magnitude twice as much as subtracting two ULPs, because of the exponent change. Neither technique is better or worse because of this, but they are different.

If my explanation doesn’t make sense then perhaps my programmer art will:

image

ULP based comparisons also have different performance characteristics. ULP based comparisons are more likely to be efficient on architectures such as SSE which encourage the reinterpreting of floats as integers. However ULPs based comparisons can cause horrible stalls on other architectures, due to the cost of moving float values to integer registers.

Normally a difference of one ULP means that the two numbers being compared have similar magnitudes – the larger one is usually no larger than 1.000000119 times larger than the smaller. But not always. Some notable exceptions are:

That’s a lot of notable exceptions. For many purposes you can ignore NaNs (you should be enabling illegal operation exceptions so that you find out when you generate them) and infinities (ditto for overflow exceptions) so that leaves denormals and zeros as the biggest possible problems. In other words, numbers at or near zero.

Infernal zero

It turns out that the entire idea of relative epsilons breaks down near zero. The reason is fairly straightforward. If you are expecting a result of zero then you are probably getting it by subtracting two numbers. In order to hit exactly zero the numbers you are subtracting need to be identical. If the numbers differ by one ULP then you will get an answer that is small compared to the numbers you are subtracting, but enormous compared to zero.

Consider the sample code at the very beginning. If we add float(0.1) ten times then we get a number that is obviously close to 1.0, and either of our relative comparisons will tell us that. However if we subtract 1.0 from the result then we get an answer of FLT_EPSILON, where we were hoping for zero. If we do a relative comparison between zero and FLT_EPSILON, or pretty much any number really, then the comparison will fail. In fact, FLT_EPSILON is 872,415,232 ULPs away from zero, despite being a number that most people would consider to be pretty small.

For another example, consider this calculation:

float someFloat = 67329.2348f; // arbitrarily chosen
// exactly one ULP away from 'someFloat'
float nextFloat = NearbyFloat(someFloat, 1);
// Returns true, numbers one ULP apart.
bool equal = AlmostEqualUlps( someFloat, nextFloat, 1);

Our test shows that someFloat and nextFloat are very close – they are neighbors. All is good. But consider what happens if we subtract them:

float diff = nextFloat - someFloat; // .0078125000
Float_t fDiff_t(diff);
// returns false, diff is 1,006,632,960 ULPs away from zero
bool equal = AlmostEqualUlps( diff, 0.0f, 1 );

While someFloat and nextFloat are very close, and their difference is small by many standards, ‘diff’ is a vast distance away from zero, and will dramatically and emphatically fail any ULPs or relative based test that compares it to zero.

There is no easy answer to this problem.

The most generic answer to this quandary is to use a mixture of absolute and relative epsilons. If the two numbers being compared are extremely close – whatever that means – then treat them as equal, regardless of their relative values. This technique is necessary any time you are expecting an answer of zero due to subtraction. The value of the absolute epsilon should be based on the magnitude of the numbers being subtracted – it should be something like maxInput * FLT_EPSILON. Unfortunately this means that it is dependent on the algorithm and the inputs. Charming.

The ULPs based technique also breaks down near zero for the technical reasons discussed just below the definition of AlmostEqualUlps.

Doing a floating-point absolute epsilon check first, and then treating all other different-signed numbers as being non-equal is the simpler and safer thing to do. Here is some possible code for doing this, both for relative epsilon and for ULPs based comparison, with an absolute epsilon ‘safety net’ to handle the near-zero case:

bool AlmostEqualUlpsAndAbs(float A, float B,
            float maxDiff, int maxUlpsDiff)
{
    // Check if the numbers are really close -- needed
    // when comparing numbers near zero.
    float absDiff = fabs(A - B);
    if (absDiff <= maxDiff)
        return true;
 
    Float_t uA(A);
    Float_t uB(B);
 
    // Different signs means they do not match.
    if (uA.Sign() != uB.Sign())
        return false;
 
    // Find the difference in ULPs.
    int ulpsDiff = abs(uA.i - uB.i);
    if (ulpsDiff <= maxUlpsDiff)
        return true;
 
    return false;
}
 
bool AlmostEqualRelativeAndAbs(float A, float B,
            float maxDiff, float maxRelDiff)
{
    // Check if the numbers are really close -- needed
    // when comparing numbers near zero.
    float diff = fabs(A - B);
    if (diff < maxDiff)
        return true;
 
    A = fabs(A);
    B = fabs(B);
    float largest = (B > A) ? B : A;
 
    if (diff <= largest * maxRelDiff)
        return true;
    return false;
}

Catastrophic cancellation, hiding in plain sight

If we calculate f1 – f2 and then compare the result to zero then we know that we are dealing with catastrophic cancellation, and that we will only get zero if f1 and f2 are equal. However, sometimes the subtraction is not so obvious.

Consider this code:

sin(pi);

It’s straightforward enough. Trigonometry teaches us that the result should be zero. But that is not the answer you will get. For double-precision and float-precision values of pi the answers I get are:

sin(double(pi)) = +0.00000000000000012246467991473532
sin(float(pi))     = -0.000000087422776

If you do an ULPs or relative epsilon comparison to the correct value of zero then this looks pretty bad. It’s a long way from zero. So what’s going on? Is the calculation of sin() really that inaccurate?

Nope. The calculation of sin() is pretty close to perfect. The problem lies elsewhere. But to understand what’s going on we have to invoke… calculus!

But first we have to acknowledge that we aren’t asking the sin function to calculate sin(pi). Instead we are asking it to calculate sin(double(pi)) or sin(float(pi)). What with pi being a transcendental and irrational and all it should be no surprise that pi cannot be exactly represented in a float, or even in a double.

Therefore, what we are really calculating is sin(pi-theta), where theta is a small number representing the difference between ‘pi’ and float(pi) or double(pi).

Calculus teaches us that, for sufficiently small values of theta, sin(pi-theta) == theta. Therefore, if our sin function is sufficiently accurate we would expect sin(double(pi)) to be roughly equal to pi-double(pi). In other words, sin(double(pi)) actually calculates the error in double(pi)! This is best shown for sin(float(pi)) because then we can easily add float(pi) to sin(float(pi)) using double precision. Insert table here:

float(pi) +3.1415927410125732
sin(float(pi)) -0.0000000874227800
float(pi) + sin(float(pi)) +3.1415926535897966

If you haven’t memorized pi to 15+ digits then this may be lost on you, but the salient point is that float(pi) + sin(float(pi)) is a more accurate value of pi than float(pi). Or, alternately, sin(float(pi)) tells you nothing more than the error in float(pi).

Woah. Dude.

Again: sin(float(pi)) equals the error in float(pi).

I’m such a geek that I think that is the coolest thing I’ve discovered in quite a while.

Think about this. Because this is profound. Here are the results of comparing sin(‘pi’) to the error in the value of ‘pi’ passed in:

sin(double(pi)) = +0.0000000000000001224646799147353207
pi-double(pi)   = +0.0000000000000001224646799147353177
sin(float(pi))  = -0.000000087422776
pi-float(pi)    = -0.000000087422780

Wow. Our predictions were correct. sin(double(pi)) is accurate to sixteen to seventeen digits as a measure of the error in double(pi), and sin(float(pi)) is accurate to six to seven digits. The main reason the sin(float(pi)) results are less accurate is because operator overloading translates this to (float)sin(float(pi)).

I think it is perversely wonderful that we can use double-precision math to precisely measure the error in a double precision constant. If VC++ would print the value of double(pi) to more digits then we could use this and some hand adding to calculate pi to over 30 digits of accuracy!

You can see this trick in action with a calculator. Just put it in radians mode, try these calculations, and note how the input plus the result add up to pi. Nerdiest bar trick ever:

pi =3.1415926535…

sin(3.14)
   =0.001592652
sin(3.1415)
   =0.000092654
sin(3.141502050)
   =0.000090603

The point is, that sin(float(pi)) is actually calculating pi-float(pi), which means that it is classic catastrophic cancellation. We should expect an absolute error (from zero) of up to about 3.14*FLT_EPSILON/2, and in fact we get a bit less than that.

Know what you’re doing

There is no silver bullet. You have to choose wisely.

Above all you need to understand what you are calculating, how stable the algorithms are, and what you should do if the error is larger than expected. Floating-point math can be stunningly accurate but you also need to understand what it is that you are actually calculating. If you want to learn more about algorithm stability you should read up on condition numbers and consider getting Michael L Overton’s excellent book Numerical Computing with IEEE Floating Point Arithmetic.

The true value of a float

In order to get the results shown above I used the same infinite precision math library I created for Fractal eXtreme to check all the math and to print numbers to high precision. I also wrote some simple code to print the true values of floats and doubles. Next time I’ll share the techniques and code used for this extended precision printing – unlocking one more piece of the floating-point puzzle, and explaining why, depending on how you define it, floats have a decimal precision of anywhere from one to over a hundred digits.

Something Positive [Comic 02-17-12] Basic Dignity
Planet IntertwinglyOrion Nebula
xkcd.comOrion Nebula
Also on the agenda: what's with his hips?
Daring FireballMLB Simplifies Pricing for iPhone and iPad Apps

Christopher Meinck, EverythingiCafe:

In previous years, baseball fans who subscribed to MLB At Bat were charged an additional fee to use MLB At Bat for iPad and yet another charge for the iPhone app. This year, we’ve just confirmed that MLB At Bat 12 will be free with your subscription, which remains at $119.99 for existing subscribers. New subscriptions will be priced at $124.99. This enables you to receive 150 Spring Training games and all 2430 regular season games (some games are subject to blackout), with no added cost for either the iPhone or iPad apps.

I like this change. Previously they had separate free and paid apps, plus the subscription fee for watching live ballgames. This is much simpler: apps are free, subscriptions cost $125. Easy. (If you like baseball, trust me, it’s money well-spent.)

(Via 9to5 Mac.)

 ★ 
Daring FireballDell’s Predicament

Dustin Curtis:

After basically admitting defeat in the consumer PC market and promising to focus on enterprise IT and “mobile services” last year, Dell has found itself in the midst of a confusing transition. It is caught between two markets that are dramatically changing. Consumer PCs are dying. Enterprise IT problems are being solved increasingly by “cloud-based” solutions using generic or custom-built equipment. The future viability of Dell’s hardware products, which already have razor-thin margins, does not look great.

Put another way, Dell has no strengths in any market that’s growing. They’re a relic.

As a side note, I found this quote from Michael Dell interesting. The Journal asked him what had most surprised him since returning as Dell CEO four years ago. He replied:

I’d say [the] rapid rise of the tablet. I didn’t completely see that coming. Tablets aren’t really new, in the sense that the tablet PC idea’s been around for a while. Obviously, more recent products have been much more successful.

“More recent products”. I’ve started to notice a trend where Apple competitors can’t bring themselves to mention the iPad by name. There are no other successful tablets. It’s just one: the iPad.

 ★ 
Dubious QualityConsole Post: Correction
Chris Karalus pointed out (correctly) that the five million unit sales is for Japan only, not worldwide.
Daring FireballHelpful Tip for Messages for Mac Beta Users With Mac.com AIM IDs

Dan Frakes:

With Messages beta, my colleagues don’t see me as being online. Fewer work interruptions, I suppose.

I had the same problem. I could see my AIM buddies, but none of my AIM buddies could see me. (Listeners of The Talk Show live broadcast last week could hear me discover this problem.) I think the problem only affects AIM users with a @mac.com AIM ID. When you upgrade from iChat to Messages, Messages assumes you want to use @me.com as your AIM ID. Apple itself treats example@me.com and example@mac.com as synonymous, but AIM does not. So what Messages is doing is logging you in as example@me.com, but you need to be logged in as example@mac.com for your buddies to see you.

Solution: Delete your AIM account in Messages’s preferences, then recreate it. In addition to restoring your visibility to your buddies, it also restores your ability to transfer files.

 ★ 
Daring FireballiPhone Mail Tip: Re-Open Most Recent Draft Message With One Tap

I seldom use draft messages on the iPhone because it’s so cumbersome to get back to them. This tip might change that. (Via Dan Frakes.)

 ★ 
Planet IntertwinglyAn economic philosophy for the modern leftist
Don’t miss Dylan Matthews’ piece on Modern Monetary Theory, which argues that deficit spending is essential to economic growth. Here’s the crux of the philosophy: This claim, that money is a “creature of the state,” is central to the theory. In a “fiat money” system like the one in place in the United States, all money is ultimately created by the government, which prints it and puts it into circulation. Consequently, the thinking goes, the government can never run out of money. It can always make more. This doesn’t mean that taxes are unnecessary. Taxes, in fact, are key to making the whole system work. The need to pay taxes compels people to use the currency printed by the government. Taxes are also sometimes necessary to prevent the economy from overheating. If consumer demand outpaces the supply of available goods, prices will jump, resulting in inflation (where prices rise even as buying power falls). In this case, taxes can tamp down spending and keep prices low. But if the theory is correct, there is no reason the amount of money the government takes in needs to match up with the amount it spends. Indeed, its followers call for massive tax cuts and deficit spending during recessions. Fighting our recession with austerity measures is failing horribly and yet favoring austerity is widely perceived as the serious-minded approach to our current economic woes. Modern Monetary Theory provides a framework for thinking about fiscal policy in a different way and shifts the Overton window away from austerity.
Ars TechnicaMegaupload's Kim Dotcom granted bail, barred from Internet

AUCKLAND, New Zealand—File-sharing magnate Kim Dotcom was granted bail Wednesday morning New Zealand time, after the judge hearing his application ruled that the Megaupload founder has no access to funds to help him flee the country.

As part of the bail conditions, Dotcom must reside at his leased Coatesville, Auckland mansion. He cannot travel more than 80 kilometers, or 50 miles, from the Coatesville residence on which no helicopters are allowed. Earlier bail applications by Dotcom failed as he was thought to have access to helicopters and chartered private jet planes with which he could flee New Zealand.

Read the rest of this article...

Read the comments on this post


Daring FireballUbuntu for Android

Newly announced project from Canonical to create Android phones which you can dock and get a full Ubuntu desktop. Perhaps the first realization of Philip Greenspun’s “Mobile Phone as Home Computer” idea from 2005? This is sort of the opposite of cloud computing. Cloud computing is “access your stuff from any device”; this is “take your stuff with you”. I don’t think this is the way to go, but it’s an interesting idea.

Jamie Keene at The Verge has a hands-on with a prototype.

 ★ 
DedefensaUSA-Inde : leçon de diplomatie par inculpabilité et indéfectibilité
Un texte rapide de R. Nicholas Burns, sur les relations des USA avec l’Inde, et sur l’Inde plus spécifiquement, et sur la diplomatie selon les USA encore plus spécifiquement, est du plus haut intérêt, – ne serait-ce que par la variété des sujets. Il importe de noter que Burns était, il y a peu encore (de 2005 à 2008), un des adjoints à la secrétaire d’Etat (Rice), pour les affaires internationales (Undersecretary of State for Political Affairs). Selon les pratiques du système de l’américanisme,
Planet IntertwinglyReport from HIMSS: health care tries to leap the chasm from the average to the superb
I couldn't attend the session today on StealthVest--and small surprise. Who wouldn't want to come see an Arduino-based garment that can hold numerous health-monitoring devices in a way that is supposed to feel like a completely normal piece of clothing? As with many events at the HIMSS conference, which has registered over 35,000 people (at least four thousand more than last year), the StealthVest presentation drew an overflow crowd. StealthVest sounds incredibly cool (and I may have another chance to report on it Thursday), but when I gave up on getting into the talk I walked downstairs to a session that sounds kind of boring but may actually be more significant: Practical Application of Control Theory to Improve Capacity in a Clinical Setting. The speakers on this session, from Banner Gateway Medical Center in Gilbert, Arizona, laid out a fairly standard use of analytics to predict when the hospital units are likely to exceed their capacity, and then to reschedule patients and provider schedules to smooth out the curve. The basic idea comes from chemical engineering, and requires them to monitor all the factors that lead patients to come in to the hospital and that determine how long they stay. Queuing theory can show when things are likely to get tight. Hospitals care a lot about these workflow issues, as Fred Trotter and David Uhlman discuss in the O'Reilly book Beyond Meaningful Use, and they have a real effect on patient care too. The reason I find this topic interesting is that capacity planning leads fairly quickly to visible cost savings. So hospitals are likely to do it. Furthermore, once they go down the path of collecting long-term data and crunching it, they may extend the practice to clinical decision support, public health reporting, and other things that can make a big different to patient care. A few stats about data in U.S. health care Do we need a big push to do such things? We sure do, and that's why meaningful use was introduced into HITECH sections of the American Recovery and Reinvestment Act. HHS released mounds of government health data on Health.data.gov hoping to serve a similar purpose. Let's just take a look at how far the United States is from using its health data effectively. Last November, a CompTIA survey (reported by Health Care IT News) found that only 28% of providers have comprehensive EHRs in use, and another 17% have partial implementations. One has to remember that even a "comprehensive" EHR is unlikely to support the sophisticated data mining, information exchange, and process improvement that will eventually lead to lower costs and better care. According to a recent Beacon Partners survey (PDF), half of the responding institutions have not yet set up an infrastructure for pursuing health information exchange, although 70% consider it a priority. The main problem, according to a HIMSS survey, is budget: HIEs are shockingly expensive. There's more to this story, which I reported on from a recent conference in Massachusetts. Stats like these have to be considered when HIMSS board chair, Charlene S. Underwood, extolled the organization's achievements in the morning keynote. HIMSS has promoted good causes, but only recently has it addressed cost, interoperability, and open source issues that can allow health IT to break out of the elite of institutions large or sophisticated enough to adopt the right practices. As signs of change, I am particularly happy to hear of HIMSS's new collaboration with Open Health Tools and their acquisition of the mHealth summit. These should guide the health care field toward more patient engagement and adaptable computer systems. HIEs are another area crying out for change. An HIE optimist With the flaccid figures for HIE adoption in mind, I met Charles Parisot, chair of Interoperability Standards and Testing Manager for EHRA, which is HIMSS's Electronic Health Records Association. The biggest EHR vendors and HIEs come together in this association, and Parisot was just stoked with positive stories about their advances. His take on the cost of HIEs is that most of them just do it in a brute force manner that doesn't work. They actually copy the data from each institution into a central database, which is hard to manage from many standpoints. The HIEs that have done it right (notably in New York state and parts of Tennessee) are sleek and low-cost. The solution involves: Keeping the data at the health care providers, and storing in the HIE only some glue data that associates the patient and the type of data to the provider. Keeping all metadata about formats out to the HIE, so that new formats, new codes, and new types of data can easily be introduced into the system without recoding the HIE. Breaking information exchange down into constituent parts--the data itself, the exchange protocols, identification, standards for encryption and integrity, etc.--and finding standard solutions for each of these. So EHRA has developed profiles (also known by its ONC term, implementation specifications) that indicate which standard is used for each part of the data exchange. Metadata can be stored in the core HL7 document, the Clinical Document Architecture, and differences between implementations of HL7 documents by different vendors can also be documented. A view of different architectures in their approach can be found in an EHRA white paper, Supporting a Robust Health Information Exchange Strategy with a Pragmatic Transport Framework. As testament to their success, Parisot claimed that the interoperability lab (a huge part of the exhibit hall floor space, and a popular destination for attendees) could set up the software connecting all the vendors' and HIEs' systems in one hour. I asked him about the simple email solution promised by the government's Direct project, and whether that may be the path forward for small, cash-strapped providers. He accepted that Direct is part of the solution, but warned that it doesn't make things so simple. Unless two providers have a pre-existing relationship, they need to be part of a directory or even a set of federated directories, and assure their identities through digital signatures. Intelligence for hospitals Finally, Parisot told me EHRA has developed standards for submitting data to EHRs from 350 types of devices, and have 50 manufacturers working on devices with these standards. I visited a booth of iSirona as an example. They accept basic monitoring data such as pulses from different systems that use different formats, and translate over 50 items of information into a simple text format that they transmit to an EHR. They also add networking to devices that communicate only over cables. Outlying values can be rejected by a person monitoring the data. The vendor pointed out that format translation will be necessary for some time to come, because neither vendors nor hospitals will replace their devices simply to implement a new data transfer protocol. For more about devices, I dropped by one of the most entertaining parts of the conference, the Intelligent Hospital Pavilion. Here, after a badge scan, you are somberly led through a series of locked doors into simulated hospital rooms where you get to watch actors in nursing outfits work with lifesize dolls and check inumerable monitors. I think the information overload is barely ameliorated and may be worsened by the arrays of constantly updated screens. But the background presentation is persuasive: by using attaching RFIDs and all sorts of other devices to everything from people to equipment, and basically making the hospital more like a factory, providers can radically speed up responses in emergency situations and reduce errors. Some devices use the ISM "junk" band, whereas more critical ones use dedicated spectrum. Redundancy is built in throughout the background servers. Waiting for the main event The US health care field held their breaths most of last week, waiting for Stage 2 meaningful use guidelines from HHS. The announcement never came, nor did it come this morning as many people had hoped. Because meaningful use is the major theme of HIMSS, and many sessions were planned on helping providers move to Stage 2, the delay in the announcement put the conference in an awkward position. HIMSS is also nonplussed over a delay in another initiative, the adoption of a new standard in the classification of disease and procedures. ICD-10 is actually pretty old, having been standardized in the 1980s, and the U.S. lags decades behind other countries in adopting it. Advantages touted for ICD-10 are: It incorporates newer discoveries in medicine than the dominant standard in the U.S., ICD-9, and therefore permits better disease tracking and treatment. Additionally, it's much more detailed than ICD-9 (with an order of magnitude more classifications). This allows the recording of more information but complicates the job of classifying a patient correctly. ICD-10 is rather controversial. Some people would prefer to base clinical decisions on SNOMED, a standard described in the Beyond Meaningful Use book mentioned earlier. Ultimately, doctors lobbied hard against the HHS timeline for adopting ICD-10 because providers are so busy with meaningful use. (But of course, the goals of adopting meaningful use are closely tied to the goals of adopting ICD-10.) It was the pushback from these institutions that led HHS to accede and announce a delay. HIMSS and many of its members were disappointed by the delay. In addition, there is an upcoming standard, ICD-11, whose sandal some say ICD-10 is not even worthy to lace. A strong suggestion that the industry just move to ICD-11 was aired in Government Health IT, and the possibility was raised in Health Care IT News as well. In addition reflecting the newest knowledge about disease, ICD-11 is praised for its interaction with SNOMED and its use of Semantic Web technology. That last point makes me a bit worried. The Semantic Web has not been widely adopted, and if people in the health IT field think ICD-10 is complex, how are they going to deal with drawing up and following relationships through OWL? I plan to learn more about ICD-11 at the conference.
Planet IntertwinglyIBM: Helping To Shape The Future Of Medicine
Turbo debriefs on some new approaches IBM is taking to help shape and improve the future of medicine.Read more at Turbo’s personal blog
Daring FireballThe Curious Case of the (Cr)apps That Make Money

More on App Store scam apps, from Trevor Gilbert at PandoDaily. Apple needs a zero tolerance policy on this crap. (Via Shawn King.)

 ★ 
Planet Intertwingly
Remember when this weblog used to be about fun links? I don't either, but I think it was somewhere in there. Well, check this out: last year when I went to PAX my most enjoyable experience was the panel "Videogames Antiques Roadshow." It worked just like you think: people would bring old game stuff up on stage, and distinguished collectors would estimate the value of the old stuff. Here are some pictures from that panel. In fact, you can see me in the second photo, fourth row center. Kind of got distracted there--the point of this post is not to look at a crowd scene that includes me. I meant to say that they brought the panel back at PAX Prime, and this time there's video. And it's now called "Retrogaming Roadshow", possibly due to trademark issues. In addition to bringing to light cool bits of history like the PCjr edition of M.U.L.E., I love the way these panels illustrate the social construction of value. Highly recommended if you've got an interest in this stuff.
Daring FireballClear

Speaking of Phill Ryu and Impending, they collaborated with Realmac Software on a new to-do list app for the iPhone. I don’t like everything about it (I don’t think apps other than games and video players should hide the status bar, for one thing), but my complaints are niggles. In the large, it’s a damn clever app, with a very thoughtful interaction design focused on letting you do a few simple things very easily.

My cardinal rule of to-do list apps is that priority should be implicit by task order. Drag more important/urgent items up, drag less import/urgent items down. Clear gets this right. Apple’s Reminders app gets this completely wrong.

 ★ 
Planet IntertwinglyPunished for being a good boy
My son is 2.5 years old. Two of his favourite TV programmes are Timmy Time and Octonauts . There are 78 episodes of Timmy Time and 52 episodes of Octonauts. There is one DVD of Octonauts, which contains eight episodes. Another DVD is due next month containing another eight episodes. This will mean that 31% of this programme is available to buy. There are eight Timmy Time DVDs totalling around 65 episodes. This means that around 83% of this programme is available to buy. To my knowledge, the BBC have aired 100% of both these series. A very quick look on popular, er, file distribution sites reveals about 50 episodes of Octonauts (96%), and I think every episode of Timmy Time (100%). I want to buy stuff, and it isn’t even possible; yet it is available for free if I download it illegally. It’s been, what, five years now? Have we really not moved on?
jwzHow public is a "public space" that requires you to sign in?

Presumably it is also behind a door marked "BEWARE OF THE LEOPARD".

The Roof Terrace at One Kearny shows why we're lucky that San Francisco requires downtown developers to provide space in their projects that is accessible to the public at large. [...]

But the only exterior hint that the terrace exists is a see-through sign etched into the glass at knee level by the front door. Once inside, a guard requires you to sign in before going farther.

Previously, previously.

Planet Intertwingly☆ A Tax Whose Time Has Come
Dubbed “the world’s most popular new tax”, this proposal has gathered an amazing breadth of support globally, including world political and business leaders. It sounds like one of the things that should be on the political agenda for any elections that might be happening this year, no? A tiny tax of a fraction of a fraction of a percent on all bank transactions that don’t involve members of the public (read: gambling)? That will raise in excess of £100 billion each year? From people whose insensitivity and abuse of society seems to have no bounds? That sounds splendid. I signed up.
Daring FireballPhill Ryu on the Rise of Scam Apps in the App Store

At one point this week, two of the top ten paid iPhone apps were outright scams. Phill Ryu has a good set of suggestions for Apple should address this. Being able to get a refund within a short window after first installing the app, for example.

 ★ 
Planet Intertwingly[2b2k] A conversation with Christopher Lydon
I’m delighted to count Christopher Lydon as a friend, albeit one I don’t see often enough. He has traveled the road, as a reporter for the NY Times, as an esteemed (and controversial) talk show host on NPR, and as one of the first adventurers in the world hybrid radio and Web. Chris and I talked last week. Here are the liners notes, so to speak, and here’s the recording.
LWN.net[$] Handset cohabitation: Ubuntu for Android
As many observers have pointed out, the phone handsets that many of us carry now exceed the power of the laptops we were carrying not all that long ago. The much-hyped Galaxy Nexus, for example, includes a 1280x720 display, 32GB of flash storage, 1GB of RAM, a 1.2GHz dual-core processor, and a number of interesting peripherals never found on that old laptop. And, of course, there is a Linux kernel running the whole thing. Given that, one might well wonder why one should still bother carrying a laptop around. Canonical, it seems, believes a number of people are wondering that; thus the announcement of Ubuntu for Android, an interesting attempt to move laptop-based activities onto the handset.

Click below (subscribers only) for the full story.