Academic Facets: News and Views from Spatial's University Community

[#1]

One of the best bits of academic life are the conferences, not just for the intellectual stuff and the meeting people, but also the shear randomness of the locations and the situations. So just after mid-night, a few Tuesdays back, saw me standing under a lamp post at a cross-roads in Turin, Italy, wondering why there was no hotel in the place marked on Google maps? Eventually down the third side street I found the place, but not before I had asked myself, if it was warm enough to sleep in the park!

 

I was Turin for the Virman 08 conference (Virtual Manufacturing) were we had a paper reporting the results of our first experiments with using CrowdSourcing for CAM/CAM reasoning. I blogged about the funding proposal almost a year ago and subsequent to that post we got a research grant that enabled us to use the Amazon mTurk site for some feasibility studies.

 

In a nutshell the idea is that CAD/CAM is full of problems that, although people find easy, computers find difficult or impossible (e.g. feature recognition, component nesting, content based indexing etc). So the research vision is to investigate if Internet based Crowdsourcing can be used to do these sorts of jobs effectively, since twenty year of research has failed to produce general, robust, automated solutions.

 

We have done two experiments; in the first we asked the “Crowd” to identify the “best” views of 3D objects from a number of alternatives. This was a deliberately ill defined task, hard to automate, but useful (imagine needing to generate small thumbnail images for each of 60,000 CAD files in an engineering database).

 

To investigate if this could be done, we  created the HIT (Human Intelligence Task) shown below and posted it on mTurk. Workers could select what they considered to be the best three views with mouse clicks. We posted 4 sets of 20 HITs, each of which contained arrays of images for 5 different components and offered $0.15 for doing the work.

   

Three results from this: 1) The Mturk workers responded very quickly, taking between 81 and  23minutes to complete each of the sets, 2) The quality of the responses was very high (only 2 out of 80 rejected) and 3) the results looked right. Arrays of images below show two of the components used and the number of times each image was selected by mTurk workers as “best” in brackets below the ID number.

 

 

Personally I think the results are excellent; when there is a clearly right answer the Crowd have selected it. And when it is ambiguous there are ties for first place.

 

Encouraged by this result we create a second HIT that asked people to arrange 3D parts into families of “similar” shapes. Again it’s the kind of ill defined task that companies with huge CAD model databases would love to automate.

 

Our HIT contained 107 parts and once again the user could select family members, and edit their selection until happy with the content (groups of one were allowed if the worker considered the part to be unique). We posted this HIT 16 time (no worker could do the same task twice) and offered $4 for the job (we figured the task would take about 25minutes and the minimum wage in the UK is £5.73 (ie $9.91)

 

Once again the response was very fast, with the average completion time for the 16 HITs being around 25 minutes (only 4 of the responses were not accepted for payment). And again the results appear to be “right”. After a bit of processing of the results we produced this dendrogram to illustrate the similarity relationships identified by the MTurk workers. This result correlated well with published classification of the same collection.

  

So at this point it looks like the mTurk Crowd is more than willing and able to tackle 3D CAD based HITs. We are currently investigating if the same approach can be used for nesting problems like 2D profile packing for sheet metal punches.

 

At this point it looks like the Crowdsourcing could be a new and powerful pattern recognition engine for many mechanical CAD applications. It would also be useful if they got hotel locations sorted out on Google maps!

  See Geometric Reasoning with a Virtual Workforce( Crowdsourcing for CAD/CAM ), P. Jagadeesan, J. Wenzel, J.R. Corney, X.T. Yan, A. Sherlock, W. Regli, Procs Intuition 08, VIRMAN Workshop, Turin, October 6th-8th 2008.