I missed the New York Tech Meetup on Tuesday and didn’t get to hear a presentation from the New York State Senate’s information officers.
Perhaps not a tragic loss.
But still this blog has a soft spot for that state’s open government initiatives, especially its RESTful, web services API.
If you’ve not ever heard of political scientist Robert Axelrod and his computer modeling of altruism in the early 1980s, immediately go and listen to the podcast of a RadioLab show—public radio’s best hour— entitled “Good.”
Do that now.
Skip to around the 41 minute mark, and you’ll hear the start of Jad and Robert chatting with Axelrod. Aside: Often when I read something interesting, I imagine it spoken in a Robert Krulwich voice to decide if it would make a worthy post.
Anyway, “Good” is the best popular introduction I’ve come across as to why we’ve evolved to be civil to others.Continue reading
Last week, the HBR blog turned its attention to the recent infestation of daily deal sites that are causing great harm to businesses.
After much study and multi-regression analysis, they’ve decided that sites like Groupon and LevelUp, are really offering … price promotions.
I was kidding about the multi-regression part. But HBR blogger Utpal M. Dholakia, Distinguished Associate Professor of Management at Rice University, warns that “price promotions are fraught with danger and are suitable only for very specific purposes.” Continue reading
I admit to being more than a little envious of those hackers who made creative use of New York City’s publicly available databases for the BigApps 2.0 competition.
Is there a way that non-programmers can share vicariously in the fun but without taxing limited tech muscles?
Yes. You just have to speak Google.
With their Public Data Explorer and the recently released DSPL formatting language, anyone with modest configuration skills—an area I excel in—can view statistical files in Google’s remarkably well designed graphing and charting app.
After spending a little time learning DSPL straightforward syntax, I decided to explore one NYC agency’s population dataset.Continue reading
The Google Public Data Explorer is a visualization app that brings life to public policy data (or really any statistics you have) through animation.
Google picked up the software when it purchased Trendalyzer in 2007.
You may a remember a popular TED conference video, “Dr. Hans Rosling’s 200 Countries, 200 Years, 4 Minutes.” Rosling was using Trendalyzer software developed by his non-profit organization, Gapminder, to dramatically display life expectancy improvements as income levels rose.
Last week, Google opened up the Public Data Explorer to accept anyone’s data— until now you could view a few sets of data from various US government agencies and the World Bank.
There’s a new data format to master as well.Continue reading
Wikipedia
I’ve yet to read Nick Carr’s latest, The Shallows, which takes a pessimistic view of the effects of writing and scanning tweets, SMSs, IMs, etc. on our neural wiring.
It’s on my reading list. Certainly his claim that our attention spans are being stunted, which may ultimately degrade our overall ability to follow more complex, non-shallow arguments when needed, has made its way into our public arguments on the Internet and always-on digital technology
In his Rough Type blog, Carr recently responded to an essay by Justin Smith that takes an opposite view. Smith, writing for Berfrois, points out that transient, non-deep relationships have been with us since the first “have a good day!” was uttered.
The Internet just turns what were trivial, meaningless interactions within our own small social groups into trivial virtual interactions with our friends, along with a much larger network of “friends”.Continue reading
While waiting for the text of the FCC’s Open Internet Order to show up on their web site, I decided it was the right time to take a quick at look Google’s fascinating Ngram viewer. So what has Google wrought this time?
From their vast digitized collection of 15 million books, they’ve analyzed 5 million and produced a frequency dataset of all phrases or ngrams up to five words in length. Even better: the frequency of a particular ngram occurrence includes a time dimension.
With the new Google Ngram Viewer, you have a cute visualization app that shows the rise and fall of phrases or expressions over the years. In the context of the FCC’s “momentous meeting” this morning, it’s worthwhile to really see how the Internet has become what we mean by communications.
I decided to compare the usage of the words “Internet” vs “telecommunications”.Continue reading
I’ve been writing lately on rating and suggestion services and their underlying data prediction technologies, which are fascinating.
What about those users (like me) who don’t completely trust the algorithmically generated suggestions that are proffered?
They can instead lose themselves in the stream of likes and comments that are displayed in the standard “recent activity” box found on the home pages of these sites. It’s a direct way to pick up ideas on movies, books, food, TV shows, and lizards.
I made up the part about lizards, but the point is that with social rating sites, anything in this world can be judged as good or bad and then become a part of the intimate information flow for the rest of humanity to see.
For example, GetGlue, the recommendation service I’ve been referring to in my posts, has an Android (and iPhone) app that lets the crowd comment on what they’re currently reading, watching, listening, or thinking. It’s really a check-in service—Foursquare without being tied to a specific physical place
With my new Yixin Android tablet now on my coffee table, I’ve become another gadget-owning media critic. Continue reading
With my request to use Google’s black-box Prediction APIs finally approved and a little time available in my schedule, I set out to see how well Google’s racks of CPUs would do against a few training sets I had in mind.
Ultimately, I was hoping to gain more insight into the question: Can software algorithms (with help from the crowd) predict what I’ll like in books, movies, web sites, and food?
To make this a manageable project, I limited the scope of my exercise to the modest problem of predicting amusing movie titles.
Wait, don’t laugh! I have some definite ideas on this subject, which I was able to compress into simple rules. For example, a number or date with an exclamation after it, funny! I’m tickled by these somewhat hypothetical movie titles:“Ten!”, “1941!”, or this real knee slapper, “22!”
I’m also similarly affected by titles with a man or woman’s name that ends in a vowel followed by an exclamation or question mark. “Ralphie?” Hilarious. “Albert.” Not funny. And titles with “Being”, as in “Being Ralphie”, are funny in a knowing, ironic way.
So how did Google’s mysterious Prediction oracle do ?Continue reading
I’m still near the starting point in my travels through recommendation services and their underlying algorithms. It’s always a great help therefore to meet a more experienced knowledge hiker returning from the other direction who can offer a better sense of the terrain ahead.
We received a comment from Sachin Kamdar, founder of recommendation startup Parse.ly, in response to a post last week on Freebase and knowledge networks that gave us just such an insight.
Kamdar’s point is that you can get pretty far—but not all the way, of course—by extracting patterns from datasets. Even a simple pattern matching algorithm can be useful.
Parse.ly, by the way, employs both data mining techniques and language processing in generating its recommendations.
So how far can you go with pattern matching and a little semantic analysis?
To find out we tried Google Sets.Continue reading
Over the last few months, recommendation startups have sprouted up—getglue, Hunch, Foodspotting, Parse.ly, Miso, Xydo (in beta), Bubbalon, etc.—to offer suggestions about restaurants, books, web sites, or just about anything in this world.
If you add in Facebook (with its like button, and lots of 3rd-party rating apps ), Amazon, and NetFlix, there’s enough of a universe to merit a service that rates and recommends recommendation services. There’s a startup, no doubt, working this out.
All share the idea that there’s wisdom in the crowd, and to various extents use stats about the mob to algorithmically classify tastes—clustering, nearest neighbor,decision trees—and then generate suggestions. There’s a nice summary of these collaborative filtering techniques in the reference section below.
What about a more conventional, common-sense approach that derives wisdom from actual knowledge of the subject?Continue reading
Yahoo has brought together 23 graduate school winners of its Key Scientific Challenges program to a tech summit it is currently holding. These doctoral candidates …
Another weekend, another hackathon. But the one that was just held in Seatle concerned itself with Gov 2.0 projects. And Technoverse favorite Tropo was there, along with open data service provider Socrata.
The winners were … ChatterCast, which monitors 911 activity in your area and sends SMS notifications, and GeoCast, which lets you learn, also via SMS, about traffic conditions within a shape you draw on a map.
Tropo scripts handled the telephony aspects for both these apps.
Congrats to the winners!
Jonah Peretti explains viral marketing
HuffingtonPost founder and “viral media marketing hotdog” Jonah Peretti spoke at the NY Viral Media Meetup last week.
Sure enough the slides from the talk have now gone viral. Here are the key takeaways, things you already knew but you just didn’t have the sense to condense into a short deck. One, viral content is spread through a network of bored office workers. Two, you never can tell what will go viral. And three, the web is ruled by crazy people.
He also dispensed valuable advice on how to present serious news on the Web, which he perfected at Huff Post. It’s something I’ll be trying to put into practice. Hint: it has to do with mullets.
Peretti’s presentation can be found in its entirety after the link.Continue reading
Google has made two tools available for relief workers involved with Pakistan’s historic floods. It’s been a bad year for natural disasters, but Google gained valuable insights in emergency management during the aftermath of the Haiti earthquake. By talking to aid workers in the field, Google discovered that up-to-date information on hospitals (available medial equipment and staffing) was crucial.
From the Haiti crisis was born Google’s Resource Finder. It’s an online, editable Web page that lets health workers update status information on medical facilities. It’s also coupled with a map for providing positioning information. Google has made an early release of Resource Finder available for the Pakistan relief efforts. Continue reading
I recently tried a silicon oracle called GetGlue. I’m genuinely impressed at how racks of CPUs can quickly navigate through an enormous knowledge graph and grab a suggestion node that matches some characteristic of my preferences. Great searching capabilities, but I’m less than excited about the results.
Like other recommendation software (say, Hunch), you register preferences by initially rating a sample list of movies, books, TV shows, and music. I told GetGlue that I liked The Breakfast Club, Goodbye Columbus, The Great Gatsby, and Nina Simone. GetGlue quickly responded with a list of predicable tips: lots of Philip Roth novels, Catcher in the Rye, American Beauty, and Sara Vaughn.
I suppose if I had lived in vault, then some of these suggestions would be novel. One quibble for the GetGlue crew: how about adding a “like, but already know about” classification.
On the plus side, GetGlue deserves credit for bringing swing band leader Jimmie Lunceford, who I hadn’t heard known about, to my attention. Thanks.Continue reading
There actually was some significant news last week in the technoverse, and it didn’t involve another episode from Mark Zuckerberg’s reality show: on July 16, Google purchased Metaweb, the semantic database company and the force behind the freewheeling Freebase.
No doubt, the semantic web has entered into your own knowledgebase during the last year.
If it hasn’t, quick go to Google: enter empire state building height in the search box. Notice that the numeric height “1250 ft. ( 380 m.)” is highlighted in the search results. Google knew to answer this query with an actual number, instead of merely returning text snippets in which those search keywords were found. This flavor of artificial intelligence comes courtesy of an analysis of the knowledge space.
In a way, Google comprehended that “empire state building” is a structure, which has an attribute or property known as height, which itself has a numeric value associated with it measured in distance units.
Impressive.Continue reading
So why was I sharing a picture of a roast chicken (food source) at a Spanish restaurant in NYC (foraging spot) with Foodspotting.com (my tribe)? I’m not getting paid for this activity, though the cost of taking the pic and uploading is vanishingly small. Am I being purely altruistic or is something else at work?
Inspired recently by Clay Shirky’s talk on his newest book, Cognitive Surplus, I took a brief tour through some of the foundational ideas behind generosity and altruism.
Our kindness to strangers may be mostly in our genes, but as Shirky and other point have pointed out, being in a network has its own benefits.
Continue reading
At Internet Week in June, I got a quick look at a 3-D printer or fabricator that was being demoed by the folks at MakerBot Industries. Their device is one part of a larger do-it-yourself movement in which both serious inventors and designers or ordinary hobbyists can prototype products in their own living rooms.
Inventing is a lonely process and development costs, though they have come down with this new crop of 3-D printers, is still a consideration. So why not crowdsource the design, protoyping, and marketing phases? That’s kind of the idea behind NYC-based Quirky. Continue reading
Harvard cognitive scientist Steven Pinker took up the mantle for twitter-reading, YouTube-watching, blog-scanning, email-reading multi-taskers in an Op-Ed in last Friday’s New York Times. In what amounts to a direct critique of Nicholas Carr’s recent remarks on hyperlinks and his new book, What the Internet is Doing to Our Brains, Pinker says that claims that our cerebellums are being re-wired to think less and react more are overblown.
Pinker scores valid points with familiar arguments from this camp. The move to the written word from an oral tradition, for example, was thought by the ancients to be a memory crutch and would inhibit good conversations and interactions. Sound familiar?
I can’t talk about whether there is a permanent rewiring of the brain, but I don’t see how Pinker and his cohorts can deny the productivity draining aspect of too much digital interactivity.Continue reading
