The Sitch with the Stitch—The CITStitch Hackathon

This is being cross-posted with iDigBio, and co-authored with Libby Ellwood and Austin Mast.

SAMSUNG CAMERA PICTURES

Internet-scale public engagement in the digitization of biodiversity research specimens, such as can be seen at Notes from Nature, DigiVol, the Smithsonian’s Transcription Center, and FromthePage.com, offers clear win-wins insofar as motoring through our 100’s-of-millions-of-specimens digitization backlog and advancing science literacy. However, developing that level of engagement presents some large cyberinfrastructure challenges, given that the community of public engagement tools has yet to interoperate “seamlessly” amongst themselves and with the more established biodiversity data platforms, such as Symbiota and iDigBio. This observation was first widely discussed at iDigBio’s 2012 Public Participation in Digitization of Biodiversity Research Specimens Workshop, which led to iDigBio’s development of Biospex, a prototype public participation project management system.

For the second year in a row, iDigBio and Notes from Nature co-organized a citizen science (CITSCI) hackathon focused on the cyberinfrastructure gaps. You can read about last year’s CITSCribe Hackathon here. The main goal of this year’s CITStitch Hackathon (Dec 3–5) was to build interoperability among projects that enable public participation in digitization in useful and exciting ways for both the public participation project managers and the public participants. Among this year’s 24 participants were developers, data managers and publishers from public participation tools as well as those who work with cool tools for data visualization (e.g., CartoDB and Zooniverse), data cleaning (e.g., VertNet, Encyclopedia of Life, Global Names, FilteredPush, Kurator, SALIX), and georeferencing (GeoLocate and CoGE).

Sophia_presentation

On Day 1, Austin Mast (Florida State Univ.) and Rob Guralnick (Univ. of Florida) welcomed everyone and provided brief introductions to iDigBio, Biospex, and Notes from Nature before lightning intros from each participant. Next, Libby Ellwood (Florida State Univ,), Austin, and Rob provided an introduction to proposed activities at the hackathon. These had been developed by iDigBio’s Interoperability for Public Participation in Digitization working group—the hackathon’s organizing committee (including those three plus Ed Gilbert, Nelson Rios, Ben Brumfield, Paul Flemons, and Greg Newman). These activities were presented as occurring in one of two tracks. The first track focused on innovative cross-platform ways to deploy and manage public participation projects, visualize and analyze progress for the project managers, and ingest data and provenance back into data management systems; this group would later be dubbed “Team Tardigrade.” The second track focused on development of novel ways to engage citizen scientists (e.g., via visualizations of individual and collective contributions); this would become “Team Honey Badger.” After this, Cody Meche gave an engaging talk on Agile development best practices and we split into our teams to develop priorities, goals for deliverables, and a road map.

What followed in Days 2 and 3 were a series of code-sprints interspersed with animated stand-ups, all fueled with a lot of coffee, hot tea, and food. In the end, each team went on to produce several deliverables involving subsets of their members that far exceeded expectations. Content regarding the deliverables can be found at the CITStitch wiki page. We have summarized the work briefly below. But as was emphasized at the start of the hackathon, successes will also be measured by the number of long-term collaborations initiated over dinner at the Reggae Shack, The Top or Andaz Indian Restaurant.

Listening_intently

Team Tardigrade

Stuart Lynn (Zooniverse) produced a broadly useful web service and data explorer for the (now 1 million!) transcriptions in Notes from Nature.
Ed Gilbert (Symbiota) and Daryl Lafferty (SALIX) produced a SALIX web service that will take an OCR text string and direct it to the correct SALIX-enabled Symbiota portal for processing; this SALIX-parsed data then can be sent to a transcription tool for proofreading.
John Wieczorek (UC–Berkeley), David Lowery (Harvard), and Dmitry Mozzherin (Marine Biological Lab, Woods Hole) produced web services for assessing the fitness for use of data and doing data cleaning, including validators for scientific name, year collected, collection locality coordinates, and measure of coordinate uncertainty.
Ben Brumfield (FromthePage.com), Greg Riccardi (Florida State Univ.), and Robert Bruhn (iDigBio’s Biospex) expanded the Biospex data model to include ledger and field book pages in anticipation of adding FromthePage as an actor in Biospex project workflows.
Finally, Greg Riccardi, Ed Gilbert, Nelson Rios (Tulane Univ.), Ben Brumfield, Robert Bruhn, and Austin Mast established a manifest file example in JSON that enables tools and project management systems to communicate about the public participation projects.

blog2 (1)

Team Honey Badger

Chris Snyder (Zooniverse), with help from Libby Ellwood (Florida State Univ.) and Rob Guralnick (University of Florida), created functionality in Notes from Nature that compares entered taxonomic names against the GBIF Name API and gives feedback to inform the citizen scientist as to whether the name exists in GBIF and the number of records associated with that name; this gives the participant a sense of the significance of their contribution.
Julie Allen (Illinois Natural History Survey), Charlotte Germain (Univ. of Florida), Sophia B Liu (USGS iCoast) and Andrew Hill (CartoDB) created dynamic maps to visualize citizen science contributions; for example, the participant could upload a dataset to cartodb.com and select subsets of the data to display the country of origin for specimens that have been transcribed. Click here for a map of countries with specimens that have been digitized in Notes from Nature.
Paul Kimberly (Smithsonian), Paul Flemons (Australian Museum), Deb Paul (iDigBio), Libby, Rob, and Austin fleshed out a proposal for a 4-day global transcription blitz organized by the major transcription centers, including its timing, name, goals, funding, and organizational structure, and scoped the functionality for its website in years 1 and later (more to come on that in future blog posts!).
And, especially relevant to the global transcription blitz website, Alex Thompson (iDigBio) produced a prototype that integrates results across different transcription platforms and generates summary results and means for further exploration using Elasticsearch.

Thank you to all of our participants—what a great experience! You can check out more photos in the iDigBio CitStitch Hackathon album on facebook.

Thanks A Million!

Last Wednesday, while many Notes from Nature folks were in a citizen science hackathon, appropriately enough, we passed the one million transcription mark. It is a big milestone, and all of us involved in the project from the science and developer team appreciate the hard work. That work helps scientists like UC Berkeley’s Joanie Ball-Damerow examine changes in dragonfly and damselfly communities over the past 100 years in California based on museum records. Her work on that will soon be published in the journal Zookeys, and your help with unlocking data makes those kind of projects possible.

In case you are curious, our Notes from Nature user snowysky transcribed data from this specimen image for our millionth record, a sweat bee (Halictus ligatus) that was hanging out on some sunflowers (Helianthus annus) when collected in 1975. We hope everyone involved in the project continues to discover all kinds of hidden treasures locked away in collections, just like this one. Now on to 2 million!

Wanted: Feedback on Zooming in our Interface

We have a lot of great stuff happening on Notes from Nature and a big thanks to everyone, and more soon, about passing the million transcription mark. But for now we’d really like some advice from transcribers on what we think is a major upgrade to our interfaces. On the Herbarium interface, we have implemented ZOOMING. On both herbarium sheets and macrofungi labels, you previously would select a region, and then get a zoomed-in version of that section. However, there were some issue with this, well noted in the Notes from Nature talk forums. The new interface uses very simple pan and zoom controls to get the label in the location and size you want.

Please try it out on the Herbarium sheet records and, assuming there aren’t any major bugs or other issues, we’ll be implementing that on the other “specimen” interfaces (e.g. the Macrofungi and Calbug collections) as soon as possible! If you do find any problems or issues, you can let us know right here, on the blog, as comments or you can log into the Notes from Nature talk forums and post there. We hope this removes a lot of frustration!

ZOOM!

A post-Thanksgiving thank you and update

A Notes From Nature Thanksgiving thank you to all of helping us to get biodiversity data transcribed and thus more easily used for new kinds of science. Whether its bees and what they are pollinating, as described by Carl Zimmer in the New York Times or examining the genomes of ancient turkeys from the Smithsonian Natural History Museum, museum collections are an amazing resource that you are helping to unlock. Thank you for the amazing effort.

As some of our devoted Ornithology ledger transcribers have likely noticed, we have finished a major effort of just shy of 3000 ledger pages, and appropriately for a bird-themed holiday in the United States, on Thanksgiving no less. There are many more ledgers at the National History Museum London that are waiting in the wings (sorry! I couldn’t resist) and we’ll be getting those up as soon as possible, along with some more information about those ledgers and their contents.

Notes from Nature and the Push To A Million

Some quick Notes from Nature updates! As you might have noticed, our numbers of transciptions have occassionally grown by leaps and bounds over the last months. What gives? The shortest possible answer is “Ornithology Ledgers” and the truly impressive effort happening there. Back in late June, we had 149,537 Ornithology transcriptions. As of last Sunday Oct. 12, 2014, that number is now 294,973. Wow. That is a lot of work in 4 months. None of those Ornithology records had been included in our total counts (that show up on our homepage), because we had originally been focusing on counting ledger “pages” not transcriptions. And we still have to generate those ledger record counts separately from our logs every few weeks and add them into the total. We hope to solve the problem with manual additions to the total in the near future. So if you see the count “jump” every few weeks, that’s why.

Speaking of counting, many weeks ago we tried to better report the “per collection” statistics, in particular the amount of effort needed to complete work on a collection. We have recently refined those numbers YET AGAIN, and we hope the current reporting is (at least) less confusing. The long story short is each record is transcribed multiple times, and that number is usually 4 times. We have plans to make this more efficient in the future, but until then, this is a workable number of replicate transcriptions. However, occassionally that number is 5 or 6 (for reasons that have to do with both history and some techology glitches). When you add all this up, it was hard to give an exact number of total transcriptions needed.

Now, if you look at any introduction to collections page – take the Herbarium project (http://www.notesfromnature.org/#/archives/herbarium) – you will see total number of images, total number of active images, total number of complete images and a count of transcriptions completed. The active plus complete image numbers should add up to the total number.  And the total number of transcrptions gives an overall assessment of effort by our citizen science volunteers. The percent completed is now calculated in terms of images not transcriptions (i.e., completed images divided by the total images).

We mention all this because Notes from Nature is closing in on a HUGE MILESTONE — one million transcriptions! We only have 113,000 transcriptions or so to go.  We will mention this more in upcoming blog posts as we make what we hope is a BIG PUSH to 1 million!

ONE MILLION TRANSCRIPTIONS

Notes from Nature Remembers

Notes from Nature is super pleased to introduce something that our citizen scientist transcribers have wanted forever – a ditto function! What does the ditto function do? Simply put: You can access and select your last few entries that you made nearly instantaneously. Yeah, Notes from Nature now “remembers” your entries! This means less scrolling through long “pick lists” (we hope). On the downside, we have only implemented it – so far – in CalBug. We hope to have this working in other interfaces as soon as is possible and we’ll, of course, keep you posted via the blog.

So here are more details about this new function. To access your last 5 entries, you just need to click Control-M while on any field. When you do hit Ctrl-M, it pulls up those past entries and you can select and click any of those entries and, voila, you have finished that field. For CalBug, where there are a lot of records from the USA, and from California, being able to quickly select those from record to record should make for more speedy transcribing. But let us know what you think! We hope it is a major improvement.

ditto
The ditto function for Country showing two previous entries


The ditto function was the work of Lisa Larson, who is a web developer and project leader at Cornell Ornithology Lab, and who attended our first Citizen Science hackathon back in December 2013. She has been great about seeing through this fantastic idea that came from a team focusing on how to improve public participation.

So, to reiterate, in CalBug, clicking Ctrl-M will help you “remember” the entries you have made earlier, and speed ability to “click through” commonly used entries. You can always access the “help text” for further reminding. Finally, if you are using a small screen, you might not have a lot of room at the bottom of the screen for the ditto function entries to show up. You might want to move the transcription tool (you can drag the main part of the tool wherever you like on the canvas) or otherwise increase screen real-estate.

Follow

Get every new post delivered to your Inbox.

Join 763 other followers

%d bloggers like this: