Partially Attended

Mendeley, a investment branch of the citation bank.

2012-01-18T00:00:00-08:00

The awesome [Rod Page][rp] has a great post on utilising Mendeley as a bank for citation data. He recommends that projects such as the [Biodiversity Heritage Library][bhl] should build on top of the kind of infrastructure that we have already created at Mendeley. I'm a big fan of this as an idea, and indeed one of my goals for Mendeley is for us to create a system that makes it easier for others to build great tools for researchers. I began down a research career, but left it quite early. Making things that help researchers gives me a huge amount of personal satisfaction. I think there are three things that we should strive for that would really help with this. 1. Better example documentation on our API. 1. More clarity around the provenance of metadata on our catalog pages. 1. A more robust framework for storing, displaying and sharing PDFs where such sharing is allowed. I was fascinated in particular by the [Worldcat tool on displaying the historical changes of an ISSN][xissn]. I aspire to being able to show provenance data on citation information in as clear a way as Worldcat have done here. Just things to keep in mind going forward. [bhl]: http://www.biodiversitylibrary.org/ [rp]: http://iphylo.blogspot.com/ [xissn]: http://worldcat.org/xissn/titlehistory

SOPA and PIPA stink, but the RWA is more dangerous to science.

2012-01-17T00:00:00-08:00

There are three bills up for consideration in the US Government which if passed will have a significant negative impact on academic research. These are the Stop Online Piracy Act (SOPA), the Protect IP Act (PIPA), and the Research Works Act (RWA).

SOPA and PIPA will have a negative impact by putting censorship controls into the hands of the entertainment industry, and permitting legal process to affect the underlying architecture of the web. These two acts are also self defeating for scholarly publishing. The RWA act is damaging as it will attempt to limit the ability of researchers to publish in open access journals. The is naked greed on the part of the commercial publishers.

The first two of these have led to a large reaction on the Internet, and this coming Wednesday a large number of popular sites, including wikipedia, will go black in protest. There is even initial evidence that the really bad parts of these bills may be deferred with the White House coming in against them, however the third of these, RWA, is getting much less coverage.

SOPA and PIPA damage the economic interests of a large number of new industries. These industries have financial clout, high profile visibility and a lot of ground level support. There should be significant and effective opposition to SOPA and PIPA. RWA only has an impact on practicing researchers, the terminally ill, the students in developing economies. Where are the obscenely wealthy champions of science and truth? They are a rare breed indeed.

These three bills have been sponsored and heavily bank rolled by industries that are doing everything that they can to retain a hold on outdated business models. Elsevier in particular has bankrolled the RWA act. Rather than playing fair in an open economy they are paying congress to tilt the law in their favour. There is nothing illegal about this so I refrain from using the term bribery, it’s just the way that congress works.

SOPA and PIPA are a call to Silicon Valley to get wise about the process of government but I don’t see any natural champions to fight against the RWA act. The Open Access movement does not have enough big money behind it to pay off enough members of government to get the bill stopped that way, the impact on the browsing public will be too small to cause a huge public intervention on the act. I’m afraid that the outlook for this act is grim.

What are the consequences if the RWA act passes?

My prediction is that this act will have a very negative long term impact on research coming out of North America. With researchers unable to disseminate their research properly, systems of research which allow the flow of information should become more fit more quickly in the information landscape. Countries that have an interest in catching up with the west, and that have less of an historic tie to overly strict interpretation of intellectual property and copyright, might find a great opportunity to definitively take the lead in the research world (go east young man!).

For the rest of us, we have to continue to build our systems in as open a way as we possibly can, and we have to continue to voice our opposition to economically and intellectually harmful legislation.

In the end the damning aspect of all three of these acts is that they promise no economic benefit, in fact, they will actively harm the economies under whose jurisdiction they fall. I would love the hear a defence from the Scholarly Kitchen about the RWA act, because I certainly don’t understand how it could be a good thing at all.

Mendeley will be joining the online protest on the 18th. Jonathan Eisen has called for a scholarly society boycott of the Association of American Publishers.

Climbing Goals 2012, review 2011

2012-01-08T00:00:00-08:00

Well, it’s been a while since I wrote up my climbing goals for 2010, and I totally missed doing this for last year, so as we ease our way into 2012 it’s high time that I do the same again.

The key goals for 2010 were:

redpoint fr 7a
boulder V5
trad E1
some trips to do some trad climbing

Well, since then I achieved the following:

onsight 6c+ indoors
redpoint 6c outside
boulder V4
a few HVS’s

So, not quite breaking through to where I wanted to be, but there is a feeling of definite progress.

2011 hi-lights:

font 6b, at fontainbleau
6c sport climb at Portland

Alltogether it was an OK year even though I have still not sent La Marie Rose, but if you contrast my training diary between 2010:

and 2011:

you can see a big hole in the middle of 2011 where I took time off to refit my kitchen. The second half of 2012 will also be very distrupted from a climing point of view (though with some very good reasons that I am very much looking forward to).

Today at the Castle I maanged to fire off a 6c, and 6c+, the 6c+ being on-sight. I’ve decided that my:

Goals for 2012 are as follows:

remain injury free
significantly reduce the amount of alcohol that I consume
fall off at least 10 routes of 6c or harder/month
redpoint 7a, but don’t stop trying harder things
get three campus board sessions or power hand sessions into my training per month

For the most part what I have realised is that what I have control over are the things I choose to do when I go training. If I can keep focussed on making sure that I don’t have any anxiety about what I am trying to climb, and that I am really pushing myself and my body, then the grades and routes will come naturally. I want to climb hard, but I don’t want to set goals that will lead me to focus on that one climb, rather I want to focus on making the journey and the process a good journey, that is using my time in a very efficient way.

example title

2011-11-21T00:00:00-08:00

Back in 2010 I was pointed at zanran, a search engine that looks for data in graphs.

I’ve been playing around with some sample searches, and decided to try drowning deaths swimming pool. Incidentally all of the results that came back were figures for young children. I guess this is because that is the group where there is the most concern, and so there is more published data about this group.

There are some interesting broad figures that come out from the data:

Figures come in on average at about 2 per 100,000 population, or 0.00002%

This is about the same chance as drowning in natural water, but at 66% more likely than drowning in a bathtub

of drownings: (natural water 25%) (swimming pool 24%) (bathtub 15%)

Cars are about 4–5 times more dangerous than swimming pools. Guns are about 10 times less dangerous.

If you are a boy in Australia, you are more than twice as likely to drown in a swimming pool than if you are a girl. I guess that this is a combination of boys being encouraged to be more reckless and girls being supervised more.

Over a year on and the site is still up. It looks like it’s still in beta, has more data than a year ago, and is registered in London.

Update on MathJax

2011-11-20T00:00:00-08:00

MathJax has come a long way since I last looked at it. There is now a CDN hosting the js files, which makes calling it really really easy. It’s being adopted by GiHub for their wiki engine.

Using it with markdown can cause a few bumps, but there seems to be a bunch of ways around this:

mathjax in markdown using some mathjax configuration and custom css
amazing implimentation in a notetaking app
php, mathjax and wordpress

Writing a block of %%\LaTeX%% like this

$$
\begin{aligned}
\dot{x} & = \sigma(y-x) \\
\dot{y} & = \rho x - y - xz \\
\dot{z} & = -\beta z + xy
\end{aligned} 
$$

will render like this

$$ \begin{aligned} \dot{x} & = \sigma(y-x) \ \dot{y} & = \rho x - y - xz \ \dot{z} & = -\beta z + xy \end{aligned} $$

Converting between dates and unix time in Python

2011-10-13T00:00:00-07:00

Going from a date to a unix time:

>>> from datetime import date
>>> from time import mktime
>>> start = date(2011, 9, 26)
>>> mktime(start.timetuple())
1316991600.0

Going from a unix time to a date:

>>> from time import strftime
>>> from datetime import datetime
>>> datetime.fromtimestamp(int("1284101485")).strftime('%Y-%m-%d %H:%M:%S')
'2010-09-10 07:51:25'

Nature, Whiskey and me

2011-10-07T00:00:00-07:00

In my earlier post on Megajournals I hinted that I felt that there could be a future in which this business model provided sufficient funds to allow a publishing house like NPG to make it’s flagship journal Nature an open access journal.

This topic came up late one evening at coasp and I ended up making a bet with Matias Piipari. I bet that within 10 years Nature would become a fully open access journal. I win three fine bottles of scotch. The date that the bet matures on Wed Sep 22, 2021. I’ve just sent the following message to the Editor in Chief at Nature:

Subject: Scientific Reports, Nature, Open Access and a future of fine whiskey.

Dear Dr. Campbell,

I am writing to congratulate you on the recent launch of Scientific Reports. 
I had the pleasure to see Sara Grimme give an excellent overview of the genesis 
of the title at the recent COSAP conference on open access publishing 
(http://www.oaspa.org/coasp/).

During the course of the event a discussion arose on the potential future 
impact of the Megajournal on top tier titles such as Nature. I professed hope 
that a vehicle such as Scientific Reports could grow in time to support a 
business model in which Nature itself could be run as an open access journal.

There was some disagreement on my prediction, the outcome of which is that 
I now have a standing bet with Matias Piipari. If Nature becomes Open Access 
on or before Wed Sep 22, 2021, ten years from now, I will win three fine bottles 
of scotch.

I will continue to keep an interested eye on the development of Scientific Reports 
and I wish Nature Publishing Group the best of luck with it.

Sincerely yours,

- Ian Mulvany

Hindawi have an awesome reviewing system.

2011-10-06T00:00:00-07:00

Hindawi publishers is a really interesting outfit. They are an open access only publisher based in Egypt. They combing a fantastic use of technology with the ability to afford a large amount of human curation over the data that they use to streamline their publication and reviewing systems.

One of their publishing vehicles is called the International Scholarly Research Network and at the recent coasp conference Paul Peters gave an overview of how their peer review system works, I think it’s genius. They take a received manuscript, look at all of the references in that manuscript, and then computationally compare overlap of the cited works with the works cited by their panel of potential reviewers. They identify up to five potential referees who have the closest discipline match, but with whom there is no conflict of interest. A review request is sent out, and the reviewers are asked should the work be published according to the ISRN criteria, or if not why not. If a qoura of reviewers accepts then the paper is published. If there is a disagreement, they all get to see each other’s comments and they have to come to an agreement on the final decision.

When a paper is published the reviewers who supported the decision have their names published along side the paper so that the community can see the provenance of the peer review.

When the decision is split I love the fact that they can see each other’s comments before making a final decision. This makes it easier for the community of reviewers to come to a concencus over what standard of reviewing to apply for the submissions to ISRN.

It’s a beautiful system, and it’s one where cantankerous reviewers can be identified quickly, and removed form the review request queue.

Megajournals

2011-10-03T00:00:00-07:00

The idea of megajournals had not really formalised in my head before, but at the COASP meeting the talk was all about “_Megajournals_”. PLoSOne is the archetype for this kind of journal, and it had not really struck me before as a huge revolution in the publishing industry, but after listening to a couple of days worth of talks on the topic I’m convincible.

Megajournals are so called because they are structured to be able to publish many more articles than has been the normal practice with traditional journals. By my count there are currently about thirty five thousand academic titles. The vast majority of these titles are not indexed for impact, many of them are homebrew, and most of them are niche. They may publish a few issues per year, they may have a handful of articles per issue, and each one is mostly supported by a small community of researchers.

The big boy journals; IOP titles, Nature, Science, Genetics, NEJM and so on, have productionised the process of publishing articles. They employ a large number of people to mange the journals and sometimes to do in-house peer review. They publish their issues weekly with a constant rolling online first systems. The big publishers have applied this model to bundle publishing across many titles. However all of these venues retain the sense of an issue. In addition many of them have explicit policies around the taste or flavour of the articles that they wish to publish.

We live in a world of selectivity in academic publishing. The journals want articles that are constrained by size, or content, or topic or perceived impact. Every journal sets it’s stall on a combination of these criteria.

The result is a high rejection rate. The work is not an appropriate topic, not impactfull enough, too long, to theoretical, not written be a friend of the editorial board, etc, etc. Nature’s rejection rate is famously greater than 98%. When a researcher gets rejected from a journal they generally have to go through the entire submission process again at another venue. It can take months for an article the make it’s way through the publishing pipeline before it gets to the final rejected state. Sometimes it can take years. Each time an article is submitted it needs to be reviewed. Reviewers need to be found, and they need to spend their time reading and commenting on that paper. If the article had been submitted to another journal before and rejected, then these reviewers are repeating the work that someone else had done. Waste waste waste waste, it’s a huge waste of time.

This is a problem both for research (delayed time to publication, multiplication of reviewing effort), and a problem for journals with very high rejection rates as costs for maintaining a high rejection rate are large composed of the maintenance of a professional staff and the processing costs of dealing with a high number of manuscripts that the journal in the end is not actually going to publish.

For a long time now people have been talking about innovating the peer review system. People have been talking about how open access is going to solve all of these issues. In the end I think really pragmatic approach is going to have a big impact, and that is the approach taken by the megajournal.

The approach of the megajournal is simple to describe. They remove the pretence of having issues, and they radically simplify the criteria for publication, asking for the most part only that the paper is scientifically rigourous. (Think about that for a moment, these journals are asking only that the papers that they publish advance our knowledge about the universe around us, not asking that they do so according to some predefined criteria. They are looking to ensure that the articles pass scientific muster, and allowing posterity to take care of the impact).i

By significantly lowering the rejection rate by not rejection based on taste, but only on correctness, you dramatically increase the efficiency of getting knowledge published. If you tie that to a business model where you generate revenue as a function of the volume of articles that you publish (author pays open access for example), then you create a nice scalable revenue stream. This worked to move PLoS into the black on the back of PLoS One.

The high impact journals have taken notice, and Nature’s Scientific Reports is an explicit move to create a megajournal that can be used to publish content that gets rejected from the selective Nature branded titles. I think this is brilliant. If the authors are willing, then for their part they can rapidly get their work published without having to endure a long resubmission process. Nature already has a large volume of content that is being submitted that they are not publishing, so the new journal should not have a cold start of submissions. The content that goes on in this new journal will be OA, leading to an increase in OA content, and one could imagine a future in which this revenue stream could supplant the traditional subscription model for nature publishing group.

There are a few interesting trends emerging around megajournals.

The first is that they by no means imply low quality as measured by the impact factor. PloS One has a rejection rate of somewhere between 25 and 30% and an initial impact factor of 4.3. That’s very good, in fact submissions more or less tripled after it received an impact factor.

Next is that when a top tier journal creates a magajournal vehicle then the reviewers of the new journal really need to be educated about what the acceptance criteria are for the new journal. Publishers from Nature, Hindawi and The Genetics Society of America all described how their reviewers continued to reject articles on the bases of more than just scientific correctness. If it’s going to work you have to be willing and able to publish work that is clearly low impact, but demonstrably correct.

Lastly, for some of these submissions are continuing to grow at a healthy pace. PLoS One now publishes something like 1.5 % of all academic articles, and it’s share is growing quarter on quarter. The implication in my mind is clear, that if this trend continues many of the small titles will be cannibalised. I asked at the conference if there was any actual evidence of this happening yet. The audience were clear to say that none have seen any impact on their journals, that there seems to be no broader evidence for such a trend, that globally the entire number of articles is growing annually at a few percent, so that so far one would not expect to see an effect. However I predict that this trend will lead to an impact on many small niche journals, and I look forward to the state of the publishing landscape in five years from now.

The rude health of Open Access Publishing.

2011-10-03T00:00:00-07:00

TL;DR OA publishing is maturing with a scalable business model that all the big publishers are jumping all over. Money will be made (but less than before), and more content will be more open. The poor lamentable nay-sayers who carp on unheard in the darkness will be forgotten, and their Cassandra-like predictions will fade to be recalled as little more than the mutterings of fools (OK, that last bit is probably opinion).

Just over a week ago I had the great pleasure to present some thoughts alt-metrics to the 3rd Conference on Open Access Publishing. The talks were recorded and my talk will be posted in time.

I wanted to write up a few thoughts that spun up out of my experience of meeting with that community and seeing what’s going on there. It seems like there is a upswell of coverage on open access right now. In the week that was in it Princeton asked it’s researchers not to hand over copyright (statement), a call to not publish with IEEE got picked up by Hacker News and the Times Higher Education posted a call for academics to not peer review for non-OA journals.

One could be forgiven for thinking that something is afoot.

Open access also got picked up in some national press, a fact that was lamented by the Scholarly Kitchen (What’s up with the Scholarly Kitchen anyway? I’ve had to be moderately careful about what I wrote in this paragraph to not stray over into mocking or insulting, but the core of my feeling on the topic is that when it comes to discussing open access publishing they are, and specifically Kent Anderson is, disingenuous about their coverage of OA in a highly negative way. I’ve met some of the contributors and they seem like nice intelligent people, so this continuos editorial stance makes me think of them as somewhat akin to the Daily Mail in the UK).

The main sentiment from the conference was that OA publishing as an industry is flourishing. There were representatives form many traditional scientific publishers, and the discussion was all about revenues, peer review models and the enormous growth that all of these titles were seeing in submissions. The biggest point of discussion over the few days of the conference was about the rise of the megajournal, more of which in a subsequent post.

What as also very clear was the diversity of philosophical approach to what OA meant from a publisher point of view. Plos focussed on their ethical standards, Hindawi talked about their scale at creating titles and running peer review over those titles in a scalable way.

Some of these trends were raised by Phil Bourne in his keynote. It’s clear that the academic has a different view from the publisher. There is still a desire to see more use of CC0 licences, and to allow wide ranging data mining on the artefacts that Open Access can produce. Phil again called for a more integrated eco system of derivative objects created on top of the literature. It’s not clear whether publishers will be in a position to do this, but OA objects should allow a competitive market place of these kinds of things to emerge. (Of course the great promise is that they should and it is by no means a given that they will).

Phil had been involved in a summer workshop on moving beyond the PDF and some of their output can be seen at the FORCE11 site. Slowly we move in the right direction, slowly, slowly, but surely.

Pete Binfield was busy telling everyone that within three years up to 50% of all content that is freshly published may be open access. Heady predictions, but form the submission figures that he showed for PloSONE not a prediction that is impossible to believe in.

Overall there was a feeling that we will see a continued and strong growth in OA published content. Back of the envelope figures were being mentioned as if they were the accepted norm and people felt that the academic publishing industry might reduce in global value from it’s current nine billion in turnover per year to a more modest two to three billion, with savings coming from the more sane cost structure that comes along with Open Access publishing. Of course the only way to tell if a change like that is going to happen is to wait it out and see.

It seems like enough traditional publishers are willing to get a toe in the water.

In the end, over the past couple of weeks the best comment I’ve seen on the current state of the OA movement has come from the inestimable John Willbanks who pointed out that it’s no longer a question of religion but of infrastructure.

SOLO11, day1, morning sessions.

2011-09-02T00:00:00-07:00

Session on engaging with peer review

This is a very nice panel discussion. For my money there are a number of key points that arose during the discussion:

discussions with the public needs to happen where the public is
being half assed about engaging the public discourse around papers, and then hiding behind peer review when you run into criticism is really bad, as for example what happened with the arsenic story nasa and science
the public needs to be educated that peer review is not binary
peer review comments should be made public (not everyone agrees)
where we have representations of papers we should look to link to conversations about those papers (trackbacks and so forth)

There is a very interesting comment in the Q&A from Martin Fenner about peer review in clinical medicine. Peer review is not as important in clinical medicine as I had assumed, far more iportant are conferences and clinical trials, and there have been cases of drugs getting approval before the peer review system completes. I found that really interesting.

One of the other interesting things about this discussion is that it avoided discussion on how long the peer review system can take.

NPG API session

Interesting, lot’s of interest in the API session. It looks like NPG is going to be working with Mashery to improve the interface to their APIs. I hope some of these people make it to the dev session that is being planned for Sunday. It looks at the moment that they are making a portal for some of their opensearch infrastructure, which is a nice thing. From the perspective of NPG, being able to see what happens through issuing API keys is a good thing, but of course it adds one layer between the developer and the data. On the whole I think it’s probably a sensible approach in today’s world as it gives you the ability to control against DDOS attacks, and when you have content that could be considered controversial by any sector of society (think climate change, think evolution), you have to be careful at some level.

I wonder if they are going to also pull in info about the open social APIs that Nature Network sits on top of? I was quite involved with rolling those out while I was still at Nature so they have a place in my heart.

Ah, interesting, NPG are putting their data onto a triple store. I think this could be important, but I just don’t know many people consuming triples compared to consuming JSON (and I say that somewhat tongue in cheek at the moment). My point though is rather that someone who is interested in institutional impact probably wants examples of some simple queries and does not really want to negotiate a triple store.

Interestingly NPG is also going to be including blog data through their APIs. They have an example of visualisations of Neuroscience blog citations. The nice thing about this data set is that it is not journal specific. I could imagine mashing this API up with the mendeley api and getting a comparison between

The dev portal will be at http://developers.nature.com

(It’s a bit weird writing this post with @spanx looking over my shoulder, - he just pointed out a typo, does that make this pair blogging?).

Science Online London Keynote, Michael Nielson on Open Sciecne

2011-09-02T00:00:00-07:00

Michael Nielson Keynote on Open Science

He rightly points out that he is probably going to be talking to the converted, so his talk is aimed at looking for resources that can help us to find answers about how to make open science works. He starts talking about an example of failure in open science. his example is an open notebook science from Tobias J Osbourne. He built up a readership of about 100 readers on a highly technical field, but he was not getting much participation, and very little feedback. He was putting in a lot of effort but was not getting much value out of the exercise. His reason, we are in a local optimum and he felt we needed a global change.

In other areas of the world we have had examples of where making a collective change has been implemented. Sweden changed the side of the road that people drive on on one day in the 1950s.

The concept, is that there is some action where if everyone changed it would be better for everyone, but you need everyone to change at the same time. There are incentives for people not to participate because there is some cost involved in changing for the individual but if the individual does not change, they get the benefit anyway from everyone else changing. This is the same kind of problem that we have with the move to open data.

This is known as the problem of Collective Action, and the definitive work on this was published in the 1960s and is The Logic of Collective ACtion by Mancur Olsen.

The problem of open science is also an example of the difficulty of providing a public good (or for an economist a club good).

A case study is regulation in the airline industry, another example is the creation of trade unions. It is difficult to exclude non-union workers from gaining from the goods brought about by a union (for example introduction of better safety measures). This means that members have an incentive to avoid paying dues. How did unions form in the first place? Historically they didn’t form in one big go, but rather small groups tended to self organise within an organisation and started bargaining with management, and the second stage was agglomeration between these groups, and between groups across companies.

Incentives changed at different scales.

Another example is facebook, there is another public good problem here, the information that you share on facebook has many of the characteristics of a public good. The way that a successful social network grows is very similar to the way that a trade union grows. You see the same pattern in successful open science projects, the ArXiV displays the same characteristics. The ArXiV started in two very small fields, and slowly expanded into other fields.

Narrowness is a feature and not a bug when you are getting started.

Elinor Ostrom (nobel prize) in her work Governing the Commons really extended Olson’s work. She is interested in looking at how you manage the commons. She noticed that in most cases the Tragedy of the Commons tends not to happen, but rather communities tend to self-govern. She asked what principles were at work in these cases?

She has a list of rules that seem to emerge. Nielson highlights three of these, and provides an example from farmland and water usage outside in small farms in the vicinity of Valencia where one expects that water usage could be a significant problem.

They have been developing a system that deals with this commons for about 1000 years, from the middle of the 15th century. It is organised into 7 syndics each one originally related to a canal or water resource. Each of these has an elected head who is responsible for monitoring problems. These people meet every Tuesday and chat, and talk about problems that have arisen. These problems are resolved by vote, and where appropriate sanctions are applied.

There is an example that is similar in the ArXiV. There is a page telling people explicitly how you should cite ArXiV documents. (every open science project needs a page like this). This is a prerequisite for monitoring and sanctions.

Nielson gives a good example on how ArXiV citations are treated in physics, and he mentions that there are different expectations in different fields about how to use these citations, some fields strongly expect them, some are indifferent, some journals tend to be hostile.

In summary open science is a collective action problem. A lot is known on how to solve these problems in other contexts, e.g. initially focus on small groups, have a collective agreement in place, have sanctions in place for breaking these collective agreements.

Talkfest, science and community

2011-09-01T00:00:00-07:00

Introduction

So this is a short TalkFest event looking at the public, communities and online science. I’m sure there will be plenty of online discussion around this topic, below are some very rough notes that I took during the event. It was very enjoyable.

The people on the panel are D. Amy Sanders from the Welcome Trust, Rob Simpson, (a.k.a OrbitingFrog), Linda Davies and Stephen Cury. The panel was convened by Alice Bell

The audience is filled with the usual suspects.

Are community groups based around hobbies a good way to reach new audiences?

Amy is giving a little bit of information about some projects that connected scientists with some offline community groups. One of the main benefits of getting involved with a hobby group is that this group is usually engaged in a social activity, and so you can engage them during their time when they are open to engagement. That’s the main takeaway. It doesn’t have to be online, being face to face and being really social can be a good thing.

Galaxy Zoo

I almost don’t have anything to say about galaxy zoo other than that it is awesome. I’m sure that most of the people here know about Galaxy Zoo. The main theme off all of the projects that have been spawned from this is to help the public help researchers. Many of the new projects are no longer astronomy projects, they cover the moon (moonzoo.org), old weather (oldweather.org), star formation in the milky way (looking within our own galaxy rather than at other galaxies, so it should fell that much closer to home (they also look at other bits in the images and they have so far found about 40 previously unknown galaxies), looking at light curves to find exoplanets (planethunters.org), papyri getting reassembled (ancientlives.org), (if you had an app that helped users to describe the nature of community science projects would that be galazyzoo-zoo, or one that helped identify the locations of zoos: zoozoo.org?).

Linda talking about the OPAL project (I don’t know anything about this)

OPAL is a community-driven research program, Linda had a hobby looking at lichens. There is an exploded pie chart, I personally am a fan of exploded pie charts. One of the big aims of the project is to get members of the public out into nature and get them naming things. This gives you an educated population and greatly increases your base for observers. They are using a host of methods to train people, new tech, a lot of work with schools. There is a lot of time spent connecting local authorities, people and scientists. They have had over 1/2 million people participating, and they have had millions of recordings. There are scientific teams across the country, those teams can lead off different research projects, these projects could be driven by the research group interest or by the community environment, e.g. one project has been looking for hedgehogs. This is very cool. They have created 40 hedgehog champions in hull who go out tagging hedgehogs (I guess she means virtually tagging them, and not spraying them with graffiti, thought that might be an interesting outcome of the project)

OPAL stands for Open Air Laboratories.

This is a very very cool project, and it is coordinating resources from across the UK, it’s just brilliant.

Stephen Curry - blogger!

Is Stephen a hobbiest blogger or not? Alice asks the question. He hates the word “hobby”, the things that people do in their spare time for fairly mild amusement? What scientists have time to do this in their spare time? An yet at the same time his hobby is blogging about science. One way to think about blogging is that it is writing in public for free, not quite as bad as masturbating in public, but sometimes it comes close (he admits that this is rather a caricature). He mentions that it is something that he does in his spare time, but something he feels compelled to do and that he enjoys doing. It sounds like William Rowan Hamilton’s justification for his poetry. It’s important to note that he does not yet feel at liberty to do this during his professional time, but that the environment is getting better. It is a good way of generating a discussion, and some of these discussion topics are of interest to his colleagues as well as to the community that he interacts with through the blog community.

QA

Moderation and quality of the data collected. Interesting, there was a recent project that involved some community participation but in the end the quality of the data for the money spent could have been replicated by on RA. Rob mentions that they try to design experiments to be robust. For Linda the OPAL project is not just about gathering data, but also about getting people to get out into nature. Within OPAL they also try to design the experiments in a robust way, e.g. get people to report number of earth worms found, rather than reporting on species type. (Rob mentioned that galaxy zoo currently has 150 equivalent full time people involved as volunteers).

Stephen Curry mentions that by engaging a lot of people you have more public impact.

What access do the public have to the databases?

For OPAL all of the project data post clean up, goes onto the national database. The live data is also immediately available (free data, yay!!). Galaxy Zoo now has data.galaxyzoo.org where all of the data is available.

Alice asks if published papers are tied back to the data, and generally the answer is yes.

What are the kinds of people who participate?

OPAL asks some personal questions, they target areas of deprivation (this is just getting cooler and cooler). They have two social scientists who are working on the project who will start publishing on this data.

question for Amy - do you choose the hobby or the people first?

The examples she talked about today was focussed on the people first, and used the hobbies as a way of reaching specific communities, e.g. wanted to engage with elderly people, ended up working with a crafts group.

did media coverage lead to identifiable bumps in increased participation?

For the OPAL project, simply yes.

Question for Stephen (from Cameron), do we have an ethical framework that can help us answer the question for what should be farmed out and what is the role of the professional researcher.

Stephen has no problem with that. His contract says something like 37.5 hours per week, but this is laughable, and anyone who works in a university knows that this is a legal fiction.

Hmm, no answer to that question.

Robb has an interesting answer, but I’m too tired to do his answer justice as it overlaps too much with some ideas that I’ve had about this topic.

what does the panel feel about the protein folding project? surely participation is it’s own reward?

Ah, the discussion is all kick off now, about people doing things for free, people who are doing

An example of where this has gone wrong, an arts graduate in Ireland asked knitters to create A4 pieces of knitting, but knitters said no.

(I mention the incentives thing again).

FarmVille does not educate anyone about agriculture

The 1 millionth lichen record was listed by an amateur, ecology and astronomy have a long history of amateur participation.

The term democratic is emerging in the discussion.

Actual amateur scientists have not been mentioned in this discussion yet, like the bio-hackers. Rupert Sheldrake is being mentioned (someone I also don’t know).

Someone mentions the polymath project. (I don’t think that the polymath example is representative, but I think the reason that I don’t think it is representative is because of an opinion that I’ve read somewhere and not because I’ve thought about it too much).

Is there scope to allow the amateur’s to level up with the main scientists?

When a new zooniverse app launches, does that lead to a drop of the other sites, Rob says that on the day of launch the other sites drop by bout 10%, but then gradually grow back to their orginial levels, there is very little canabalisation.

Someone mentions the question of what is science for, doing science in your garage may not help the world, as it may not get out of your garage.

Jack Stilgo .. oh, hey, we are finished now, and off to the pub!!

ArXiV at 20, a brief review.

2011-08-30T00:00:00-07:00

I just read the brief article by Paul Ginsparg about 20 years of the ArXiV. I think the article is a must read for anyone who is thinking about scientific communication. It’s short, and very readable.

The things that stand out for me from this article are the following:

There are real costs associated with running services on the web.

Ginsparg originally imagined that the service that he created could run automatically and that he would be able to get back to his research projects very quickly. The enterprise ended up being a full time job for 20 years. The overhead implicit in doing peer review, or even minimal quality filtering had prevented the addition of features into the site. This is not much to the disadvantage of the ArXiV, as Ginsparg points out it is perfectly OK for such services to live outside of the ArXiV, but yet be tightly coupled to it.

What we can do on the web has been well imagined for a long time, we just don’t seem to be doing it.

The ArXiV is still running some of the original software. The core of the services has not changed radically over the past 20 years, but still, some of the original ideas that Ginsparg had all that time ago have not been widely adopted on the web of scientific communication yet. These ideas are easy to state - interactive data graphs, automated filtering and peer review, semantic markup, the end of the journal, web structured documents that go beyond print originated formats. Perhaps this revolution in scientific communication, in spite of the advance of web frameworks, is a long cycle revolution.

In spite of researcher anxiety, these tools bring real value.

Fields which slowly and cautiously adopted the preprint system have never looked back, and have prospered because of it. His point about fields where a researcher can get gazumped after announcing a result at a conference but getting tied up in the peer review system to lose out to a rival who publishes faster based on the conference result screams of a disfunction in some areas of science. It might not be a revolution that we are engaged in so much as a gently, but persistent, annealing of the system. Bring it on say I.

Serendipity, a chance encounter (obviously)

2011-08-08T00:00:00-07:00

Tonight there was a lovely sameas event on the topic of serendipity. I’d stared a blog post about this back in May as a response to to a post that Frank Norman wrote about enablining serendipidous discovery in the digital library. Well, that post kind of lingered, malingered in my drafts directory, so I thought I’d best get on get something out there, spurred on by the event tonight.

I studied in Edinburgh for a few years, it’s a beautiful city, and in the heart of the old town Victoria Street sweeps in a steep curve down to the cowgate.

On the corner there used to be a curiosity shop and I remember so well it’s opening times, painted in a cursive script on the side of the building “Open by Serendipidy”. It logdged strongly in my imagination, I can remember the first time I passed by when the shop was open. Inside I discovered an emporium filled with victorian glass eyes, 1940 photographic equipment salviged from fighter aircraft, a thousand pieces of bric a brac, each with it’s own personal history, a little hidden from us now.

I never had the chance to find that gem in that shop that triggered some hidden or half forgotten connection, but the place was filled with that magic of possability. You couldn’t help but feel all of those stories waiting to be rediscovered.

Serendipitous discovery has at it’s heart has a little of that magic. It tickles some part of our brain, it feels like a little revelation, a personal discovery, a moment where previously unaligned things in the world sit now in a new relationship to each other.

I’m not a researcher, but I was very fortunate to spend some days with some very smart people thinking about this topic. We looked at how one could use social media to introduce serendipidous connections between peoplae, but the findings are quite general. What’s might be neat is that there is a formula for serendipidy, and you can measure it in WoW’s!

The formula came out of research into machine vision, and it uses a nice applicaiton of Bayes’ Theorm to take into account how we re-adjust to the new and how a change in a stimulus fades as it becomes familliar. The original paper looked images, and tried to determine what chagne in an image would be considered most surprising by an observer. I thought that the same mechanisim should surely be applicable to netowrks. If you could weight the informaiton content of a network by appying some measrue to the connections, then this formulat might be able to tell you which rewiring of the network for a given node would be the most surprising connection for that node.

Our social netowrks are not composed of simple connections. We have many different connections of many different types. I wondered whether one could mine into the hidden or partially obsured ties between people and bring out the make clear to them the surprising kinds of connections. For example, at a conference about a particular topic, there is no surprise that all of the people there are interested in that topic, but where some of those people are connected by some link that has nothing to do with the topic at hand, you might be able to bring out a surprising connection, you might be able to enable a seridipitous moment. This would not be discovery, not saying to someone somethng they might reasonably know, but rather uncovering a hidden, perhaps even an uncanny link between people, perhaps even a link they may prefer to remain hidden or uncovered?

The discussion tonight was really fun. I had thought about this topic a lot over the past few years, but only thorugh the lens that I’ve laid out in this post. The talk tonight made me think a little afresh about this. Unsurprisingly, of course, it’s complicated. That beautiful equation is only a model. These netoworks that we build online again only models. The mathematical structures are loose nets, shaow representations of social realities, capturing only whisps of the structures they aim to represent.

We like to be surpried (when the surprises are pleasant ones). We like to tell stories, to find the narratives of ours and others lives. Serendipity has a powerfull hold over our imaginations, and I think perhaps because it is the making of a very good story. That connection that instantly brings us closer to another person. And yet what are we doing when we talk to people, other than probing, looking in a very directed way (though perhaps unconcsiouly) though the shape of what they tell us for a connection, a moment where we can go “ah-ha”. We pass over the mundane and find our way quickly to the delightful, the conneciton, the turn in the story, the revelation.

My thinking about all of this started with a desire to find ways to use technology to make people happy. I’m sure that is still a goal we can aspire to, perhaps by progromatically intorducing a little more chaos into our lives. I’ll have to leave now, these threads to others to follow up. Perhaps I’ll be surprised by what I find out here in a few years.

I think that the web is in good hands. Ciro Cattuto, and researchers like him, are doing amzing work looking at how people are connected in meatspace, on the web. Lanyrd arrived like that, out of nowhere, out of a honeymoon. These people are building some very powerfull ways of brining people together. They are doing it with a dash of delight. I hope that over the next few years there will be many many places on the web that will be a little like that shop I still remember so fondly from my days in Edinburgh.

Robert Bunsen's Birthday

2011-03-31T00:00:00-07:00

Google’s homepage cartoon today tells me that it’s Robert Bunsen’s birthday. He is famous for the invention of the burner named after him, but his contribution to our understanding of the universe around us runs much much deeper than that. He co-created the science of spectroscopy with Gustav Kirchhoff. This is something I learned about when I was living for a while in Heidelberg, one of the university building’s along the Hauptstrasse is named after Bunsen.

The invention of spectroscopy was a huge step in unlocking the structure of the universe at the largest and smallest scales. It led to the experiments that laid the foundation of atomic theory. It enabled us to see what the stars are made from. Before spectroscopy we could see the stars, but in a way, after it we were able to taste them, to understand their properties and compositions.

The philosopher Aguste Compte wrote in 1835 that:

On the subject of stars, all investigations which are not ultimately reducible to simple visual observations are…necessarily denied to us… We shall never be able by any means to study their chemical composition.

It was spectroscopy that blew away that limit. Spectroscopy is the science that is used to determine the red-shift of galaxies, and so it is also used to measure the extent of the wider universe. It turned light from a pure substance to a code of substances.

I often wonder now about the limits of knowledge, about whether we can ever understand experience in an analytic way, whether the Hubble horizon of the visible universe shields from us strangeness that we can never know about, whether as we look at the universe on smaller and smaller scales information truly becomes the only thing of matter, whether the complexity expressed in every simple act in the changing of the seasons is a complexity too much for us to bear.

I’m often tempted to side with the view of there being ultimate unknowables. And then I think a little about spectroscopy, and how such a simple change in our point of view changed so much of what we could view and understand, and I hold back a little more.

Happy birthday Robert Bunsen!

Sits meeting notes, November 2010.

2011-03-30T00:00:00-07:00

2010-11-04 sits

Right, the meeting is starting, it’s an open meeting, so we don’t have a specific agenda, but as the day goes on topics for discussion will emerge, and these will be time-boxed for discussion.

As the participation list is available on the eventbrite site, I’m not going to capture that here.

Some topics from the last meeting

signatures on digital objects
correct identifiers on digital objects
author identifiers
lightweight languages
toolsets
sword, sword2

Some potential topics for this meeting

authentication and access control
rdf and linked data within institutions and repositories
data curation
reasonable workflow components, best practices
lightweight tool sharing
web archiving
sustainable storage
tool sharing
data curation, big data, and what the concept of curation means for repositories


Actual topics discussed
Authentication
Linked data
Web Archiving
Identifiers for people
Microservices
SEO and search engine optimisation
Lightweight Languages

topic discussions

Authentication

The first option that is discussed is shibboleth. Generally authentication breaks down according on one of the following mechanisms:

shib
non-shib
ip based

One of the questions is how does service host have trust in 3rd party services, and understand whether that other provider is doing a good job?

A problem with ldap is that groups within ldap could go out of date.: question: has shibboleth provided support for services yet?; answer: it’s an interesting question

Are there approaches on service to service authentication that have worked 3 concrete examples:

j2k service, it retrieves content over http
integrating with JOVE, pass in to it a url source, and if that content is protected it will be the same issue
fedora

So there are social, and technological problems

How does one manage individual level access on a lightweight basis, for instance, allowing people to pass something off to an iPhone. How do you enable content owners to set the access permissions of their own data.

If you deliver all of your content with Drupal, you can just hook into drupal’s authentication systems, and this makes it look to the end user that they are just managing a part of their site.

So there are a couple of things going on here:

institute to institute
institute to end users around content held by the institute
data owner within institute to 3rd party applications for that user, e.g. iPhone apps
data owner within institute setting permissions around their own data

An interesting use case is how you allow anonymous local users on a campus wide system to access content. Shib2 has a way of allowing an ip address to skip the authentication step, it’s a pretty static list. This use case was designed for kiosk machines

There are ezproxy campuses and non-ezproxy campuses. This use case is slightly different from the normal experience of a web user because when they go to a 3rd party service we need to figure out a way that would enable them to assert the university rights at that end point.

takeaways from this talking point:

think about pushing the authentication down into the stack, so that the front layer can be plugged into a number of different areas? (I didn’t capture that point well)
articulate how openauth does or does not work within institutions (what does saml give you that oauth does not give you?)
are there any examples of universities that are using openid?
think about creating a microservices driven layer for multi-authentication systems

Linked data

What is happening and what is not happening, what could easily happen, and why is it not happening?

In blacklight search repositories there is a lot of content aggregating there, and these could be exposed as rdf. I ask what is the advantage or disadvantage of rdf over opensearch. This is answered quite nicely, linked data is about known content, and opensearch is about finding content when you don’t know exactly where it might be. Think of linked data a bit like rss.

Another question, is there other content that we should be thinking about pushing out too, is there local content, what would that look like, and how would that help in the open world?

Is there a spec for exposing triples as linked data. There are hundreds of specs, and many different ontologies. The best thing to do might be to wait and see what the big players are doing. A good example is Google’s recent adoption of GoodRelations.

One thing that really bit Southampton is to think about how it is going to be consumed, and ideally consume it yourself. A good analogy is to create a user interface, and never use that user interface yourself. It’s the only way where one will find the minor but really annoying errors.

Something to look for are javascript tools and restful apis for playing with the semantic web. Jenny Tennison is working on some of these tools like redfquery.

Another issue is what namespace do you link to, and where is the trust. (I’m in sameas, yay! http://sameas.org/html?q=ian+mulvany&x=0&y=0)

How do you decide what you want to link to?

some ontologies that might be useful

The BIBO ontology seems to be a good first place to look, and here are a bunch:

Action Items

have noted that we need to identify promising formats and vocabularies
a low hanging fruit is to have a community effort to identify, even at the predicate level, what terms to use (there is a lot of discussion going on in the UK between various groups, but there is no formal …
we will set up a freebase page for collecting ideas around this

Web Archiving

From the maemento perspective, an archive is no different from the rest of the web, it just has content from the rest of the web. That includes CMS’s, these are just archives systems, as they have the content from the past sitting in their databases.

Issues:

versioning requirements
accessibility

As a starting point it would be interesting to see what the Duraspace people are thinking.

It seems like making a repository a time gate for itself is a great idea. Dspace, however, does not do versioning. As a part of the re-architecting there is a hope to get versioning into Dspace via the fedora component, but it is a little way off.

Hydra would be dependant on the fedora component, but it is not implemented at a high level yet.

What % of revisions in repository systems at the moment are ones that could be exposed, compared to ones that are just typos?

This is a decision that one has to make in general, there may be legal restrictions, for instance. But if you do expose them, then doing so in a way that is browsable is probably a good idea.

An interesting question, if something like google becomes maemento aware then it’s likely that this will take care of this issue.

There seems a lot of years between now, and when google is going to provide time aware content.

There is a difference between doing a historic search, and currently available history.

An interesting thing might be to be able to get a list of recent uris from an archive, and find a way to preserve the content at those uris.

There is a “wayn” protocol for handing over web archives from one place to another.

A question comes up for how do you archive rapidly moving web sites. There is no good approach for this yet. Transactional archiving is an approach that might be investigated.

Action Items

if you want to be well behaved, it can be hard to archive social media sites (one could look at consuming feeds)

Lunch

Now we have a discussion about what we will talk about in the evening session.

Identifiers for people

In terms of author identifiers, Southampton have gone for a local approach, and are creating linked data identifiers. Where two people are the same, use sameas to link them. They do make them available.

The ORCID project is going to go and push ORCIDs onto people that publishers know about. This won’t happen immediately, but will get better as time goes on. Disambiguating authors is costly, so there is a large advantage from doing this as early in the publication process as you can.

ORCID will also act as a registry, so one can lookup things like a REPEC and other identifiers that an author might have.

The big question is whether ORCID should just be an identifier, or whether it should contain information like bibliographies. The issue is if it is an identifier service only, how will it get paid for.

Right now thee is a testbed, the code from Thompson has been submitted, does not cover every use case yet.

There is a hope that this will be available in 2011 at some level. There is a big stakeholders group meeting in London on November 18th.

There will be APIs for deposit and search, as well as a web protocol

Independent researchers will be able to register themselves.

There will be provenance for who made different assertions, whether, for example, a publisher or an institution the independent researcher.

Some questions: has there been any discussion around assuming that this needs to be a central vs distributed blob of information?

isni is another initiative, it’s an iso standard. A number of music. isni’s require that you have published something that people consider to be authoritative. The names project seems to be working with the isni project.

There is work ongoing to test ORCIDs against Dspace.

Resources

Microservices

What do we mean by micro-services? It’s becoming a buzz word because it is identifying an emerging pattern.

There seems to be some confusion over microservices as just being the CDL implementation in contrast to a general architectural approach.

I’m going to skip out and talk to my wife now ;) OK, I’m back from the conversation with my wife.

Resources

CDLIB

We are now onto the final session.

SEO and search engine optimisation

Dspace is working with google scholar, and Anurag at google. Essentially there are a list of tags that google scholar wants, and if you fill those out then google scholar will give you more coverage.

Scholar has a citation meta-tag schema, and so the exercise is to map dublin core to that schema. Many of the local repositories (what they really want are the pdfs, and have metadata around those pdfs). What google scholar want is totally different from what Google news wants, and what the main google search engine wants.

They already had google site maps, but google scholar don’t use those.

You ask scholar to crawl your site. You email Anurag.

An interesting question is how much traffic comes in from google, there is data that about 90% of the repo traffic comes in from google scholar, and more and more of the faculty are now using google scholar over web of science, so this is really really important.

What’s interesting is that google scholar are trying to encourage people to use the de-facto Google Scholar standard

An interesting use of testing is to use cucumber and a subject specific expert to write tests like:

“when I search for blah” “I should get back search result blah”

There is an interesting conversation about how would this community propose as a better way to organise metadata on the web, and potentially look to propose something in RDF.

Resources

Actions

Perhaps take this issue to NISO, to determine whether there is space for publishers and repositories to push back against google for adoption of better metadata tags. An example would be to propose possibly RDFa (perhaps under the bibo ontology)

Lightweight Languages

A discussion ensues about adopting new tools, and how to convince IT managers to adopt new tools.

security holes will almost always come from customisations
setting expectations, and having a regular patch cycle
ensure that there is a good shared install base with the same dependancy trees
can you turn something old off
log inbound requests for software stacks
bundle your stuff as a war file and make it pretend that it’s a java program

We have not talked about developer retraining. Why should people learn a new language?

Resources

Squid-cache

mini-review of "Measuring the User Experience on a Large Scale"

2011-03-29T00:00:00-07:00

Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications by Kerry Rodden, Hilary Hutchinson, and Xin Fu

Rodden, Hutchinson and Fu describe a framework for measuring the user experience of web apps through mining server logs. Since they are based at Google one assumes that the framework that they are describing has been battle tested.

They focus on metrics that measure user centric aspects of the online experience, in contrast to business centric (such as PULSE metrics: Page-views, Uptime, Latency, Seven day actives, Earnings). They maintain that the metrics they propose are better suited to products that have been launced, rather than being used in the design stage. They maintain that these metrics should be used in conjunction with other measures, such as direct user studies.

They name their metrics HEART metrics: Happiness, Engagement, Adoption, Retention, and Task success.

Happiness is measured through user surveys. An interesting insight is that the iGoogle team introduced a new design, and saw a drop in happiness scores with the design, but these improved over time, indicating that the initial drop was due to change aversion. They measure this score weekly through in-app surveys that can be deployed at scale, and so they have a historic record of user happiness.

Engagement is some activity per volume of users, such as average number of visits or actions per user over a time frame. This is best reported on an average per user basis.

Adoption and retention are rather self-evident metrics.

Task success seems more problematic to track, as one needs to set up the metric around task success beforehand, and then A/B test a set of potential variable scenarios.

Overall this is a nice little paper that articulates well the challenges around understanding user interaction with one’s site. The clear challenge remains deciding what one’s goals are, and then coming to an agreement about what the signals are that can be tracked that will inform you about whether your goals are being met.

Our gospels define us

2011-02-27T00:00:00-08:00

I’m reading a fascinating book about the Wittgenstein family. It’s called House Wittgenstein, a family a war. The book main focuses on Paul Wittgenstein, the left armed concert pianist, nonetheless there are many things detailed in this book that have dramatically informed my opinion on both Ludwig Wittgenstein and his philosophy.

Two things I want to mention here, firstly much of Ludwig’s approach to life was influenced by Tolstoy’s work The Gospel in Brief (The parallels between the two works are drawn out very nicely here), secondly Wittgenstein’s close family constantly refer to him as being a saint in their personal correspondences.

When you read the Tractatus it’s hard not to notice the rather spiritual aspects of the work. I was originally introduced to the work in the context of the logical positivists and the Vienna Circle. Back then I recall that Wittgenstein himself didn’t have a great deal of time for that movement, and seeing where part of his main influence has come from this no longer surprises me.

By the account in this book, Wittgenstein seemed to attract a rather cult like following at Cambridge. His family thought of him as being the rather slow and stupid sibling, and were rather embarrassed that England had taken him to heart as a great philosopher.

On another note The history of Rome podcast presented a very interesting point of view a few episodes ago on one of the driving motivations for the spread of Christianity. This happened within the context of a roman culture where most of the population were slaves. Christianity offered salvation to all people, including slaves, and so for a time when there was great uncertainty in the world, and no other structured religion really offering anything to many people, Christianity spread rapidly. The times drove the adoption of their gospel.

I think now to the last 80 years. Wittgenstein’s work is a powerful one, and the author displayed aspects of sainthood in his persona, yet his work has not gained widespread dissemination. His fame has, but not so much his work. One of the most widely spreading “Gospels” of this age is that of scientology, which seems to be a fame driven message. I believe that the gospels being chosen now are a good reflection of the times we live in.

Rat in the Snow

2011-02-06T00:00:00-08:00

I’s a beautiful morning in New York, the sun is shining, and the snow blankets central park. Yeserday the rain and mist made it feel like a bit of a shit-house, but today it’s a fantastic day.

I brought my climbing boots on this trip just in case I might find a few hours to squeeze in an attempt on the polish traverse on rat rock, and this morning I had my chance to get up and see what state the rock was in. Sadly the polish traverse problem had a thick covering of snow on most of the holds, and the other holds were wet.

Around the corner the slightly overhanging wall on rat rock wa in the sun, and there were three desperate, but dry holds.

I messed around a little on the polish traverse, even though the holds were wet, and to my delight the key rail on the crux felt much nicer than it had ever before. The shape conforms very well to the mid-sized campus rungs that I’m training on at the moment, and so my right hand on the rail felt really comfortable probably fr the first time ever. Pity that the rest of the route was totally unclimbable.

I trotted around to the sunny side, and having a little bit of time to kill I pulled on my climbing shoes and strarted playing on those three desperate holds. They are pretty despreate for me, and there is an established start to a problem on that side that involves pulling on two of them and making a slap to a flat jug. I hadn’t been able to do that start that last time I was here in August of 2010, but since I didn’t have any options for trying anything else I just stood there in the sun working those two holds.

I had been able to do that start when I lived in New York, back in 1999 - 2001, but only every now and again, and only when the conditiions were pretty perfect.

It was georgeous, just standing there in the sun, pulling on every few minutes. After the fith try I could get some movement on them, they had gone from feel totally unusable, to actually not that bad. After about ten tries I got the move. If more of the face had been in condition I would have usually stopped trying after a handfull of moves. It was a really great lesson in focus, and in persistene.

All that rock, only those three holds usable, and a little bit of a relaxed approach brough a nice little start to my day :)

Turning the Physics ArXiV into an Open Peer Review System.

2011-01-04T00:00:00-08:00

Axel Boldt posted an interesting short paper discussing how to turn the physics ArXiV into an open peer review system. It’s a short read, about three pages, but if you are familiar with the problems around peer review then you can just jump to part three of the paper which is a little under a page.

The solution proposed is to create a new role of editor on the ArXiV, and allow anyone to propose their paper for review. An open, almost endless, review process could ensue if scientists wanted to contribute time to review items. The editor has to choose who gets to review the paper, and this layer of peer review would require some maintenance. An extension to the idea might be to allow anyone to peer review a paper that was in the “reviewing pool”, and then attach reviewing profiles to the people who had done reviews. Those profiles could include information about the connections between reviewers and reviwees, and one could imagine an editor refusing the peer review stamp if those explicit connections between reviewer and reviewee were too close. With another tweak, one could imagine assigning points to the reviewers, and if one has enough review points, then you get to somehow jump the queue when you submit a paper for review.

I’d say Axel’s idea is a good starting point. Many of the social gaming features that you would want to incorporate into a site that allows social reviews are understood, and if you built it right it could work. I think the success would depend on getting an initial stable or high-quality editors associated with a “journal brand”, that would attract submissions, and in addition you would want to build a fairly seemless system, but it’s not implausible to see that such a system could be built.

Cancelled :( Drinks with Chris Wiggins.

2010-12-20T00:00:00-08:00

CANCELLED

Due to flight cancellations out of NY, Chris won't be making it over to this side of the pond, so sadly we will be calling the event off.

London is a great city to live in if you are interested in technology. It has to be one of the best cities in the world for people working on the interface between science, tech and te web. Tonight I’ll be going to the awesome Same As Christmass Quiz, and tomorrow night Chris Wiggins will be passing through from New York, and there will be a chance to meet up with him for a pint and a chat.

Chris and I met a couple of years ago while I was still working at Spriner. I was visiting my old haunt at Columbia University, and I dopped in to see my old professor Ed Spiegel. Ed told me about Chris, and I dropped in to say hi. Chris has done some brilliant work on networks and their application in the biological sciences. He has also helped run some of the New York Nature Network events, and earlier this year he ran a hack day for students in New York.

He’ll be in London for one night only, and is interested in meeting up with London based Data Geeks. If you are around and you want to pop in for a pint and a chat, we will be propping up the bar at the Jeremey Bentham on University Street from about 7pm onwards tomorrow night.

Wrangling image metadata

2010-12-13T00:00:00-08:00

I’m trying to move away from iPhoto because I have lost too many pictures over the years from having multiple versions of an image lying around, from metadata being plopped all over the place. It is time to move to a more sustainable simple way of naming foldering and tagging my image archive.

In the course of looking at this I have been playing around with Lightroom, and I want to ensure that I can add tags at a system level on my mac, and have them show up in Lightroom, and at the same time have Lightroom tags discoverable through spotlight searches.

There are obviously many strategies.

What I have discovered so far is the following:

When you write metadata to a file from Lightroom, it writes that metadata into an associated xmp file.

When Lightroom reads metadata from a file it will read it straight from the exif profile in the file.

exiftool by Phil Harvey is the best commandline tool for interrogating and manipulating metadata at a low level.

If I add a keywork in Picassa to a Raw file, nothing happens on disk, until I save my changes, and then Picassa creates a jpg version of the Raw file, and adds the keywords into the exif under both the “Subject” and “Keyword” tags. I guess I should just not use Picassa to edit keywords for a while. In fact from this description you see that Picassa does some hiding of the original versions of the files and you can’t see the originals anymore. I don’t really like this.

Video workflow with a mac using iMovie and capturing with a Lumix GF1

2010-11-14T00:00:00-08:00

I’m figuring out the best way to shoot video with my Lumix GF1 on a mac using iMovie. The GF1 offers two main options for video capture, HD JPEG, and AVCHD. There is a great post describing the pros and cons of these two formats here , and based on that I decided to experiment with AVCHD. After playing around a little I found a few oddities. JPEG capture writes the files onto the device as .MOV files inside the relevant directory, along with any images that you capture. By manually looking through the memory card from my camera I found these files in /DCIM/102_PANA, where the number before the PANA will be specific to which set of images you are capturing.

When I tried adding the .MOV files directly to iMovie by opening the camera as an attached device from within iMovie iMovie didn’t see these .MOV files. I was only able to add these files to iMovie, either manually, via image capture, or via adding these videos first to iPhoto.

On the other hand, when I look at the device from iMovie, iMovie does see the videos that I captures as AVCHD, and imports them. From the post that I read to see what the differences are between these two formats I was expecting videos that were captures as AVCHD to be much smaller that JPEG videos, but on first examination that was not the case. I shot about one minute of the same scene in both formats, and I got the following file sizes:

JPEG HD capture: sample.MOV 293 MB ACHVD capture, after import to iMovie: sample.mov 383.7 MB

The ACHVD file is on the camera device in the following folder: /PRIVATE/AVCHD/BDMV/STREAM/sample.MTS and has a size of 129 MB.

There is obviously some conversion going on during the iMovie import. iMovie will also not directly import the .MTS file, but needs to be looking at the camera as an attached device.

I decided to try converting the .MTS file via handbrake, and import the converted file.

original sample.MTS 129MB
handbrake converted file sample.mp4 20.6 MB
iMovie imported mp4 file sample_converted_imported.mov 368.6 MB

So I gained a little, but not very much at all.

Taking this clip and exporting it as a HD movie using iMovie I get the following file: imove_export.mov 119.1 MB, and this is actually pretty close in size to the original .MTS file stored on the camera. I can use handbrake again on this file to get an mp4 file of the size 20.9 MB.

I don’t know what is going on under the hood when iMovie makes it’s internal conversion from the .MTS format to the .MOV format, so I don’t know whether going straight into iMovie from the camera, or going via a conversion with Handbrake will end up with a better quality movie at the end or not. I’m lazy, so I’ll probably use the following workflow, suck up files using iMovie, at the same time store the .MTS files locally in case I want to clear out the iMovie archive, and work on the originals later. That seems to give me the best balance between a low friction workflow, and having manageable files. When I get some sample video out, I’ll post some of them on this blog.

San Francisco, and science hack day discussions

2010-11-02T00:00:00-07:00

(This was from Sunday)

In an attempt to get over the jet lag, myself and Dave headed over to San Francisco yesterday, we had a good walk around, got lost twice, and then met pup with the lovely Ariel Waldman. I had to drop off some mendeley schwag for the upcoming San Francisco science hack day that Ariel is organising (we are giving a little sponsorship, yay!). It’s going to take place in two weeks, and it looks lie it’s going to be an awesome event.

These science hack days grew out of a panel discussion that Ariel let at the last south by southwest conference. Jeremey Keith organized the first one that took place in London. I was at the London event, and that rocked,

Some key things for hack days, not too much sugar, keep the beer for later, near the presentations and find a good way to capture some of the context from which the hacks emerge, the description of the hacks from the London event are a good reminder if you were there, but not so great for people looking in from the outside.

We got chatting, and of course that just threw up a whole bun of people that both Ariel and know, she was about to have dinner that evening with argon smith, I’d just met him two weeks ago in oxford, small small world.

After that Dave and I headed around San Fran for a bit of exploring (science fiction and fantasy bookshops, climbing walls, comic sea lions, and antediluvian coin operated fortune telling machines all featured in a rather event packed evening), then back on the cal train to our hotel and a minor battle with the jet lag.

DLF opening keynote, beyond buckets and boxes

2010-11-02T00:00:00-07:00

(notes taken yesterday)

The keynote is starting, the DLF has grown by 15 new member organisations, and this is one of the best attended events that they have run, the room is packed, i didn’t realise that there were so many people in the hotel, its looking like its going to be an interesting meeting.

The original vision for this group is to create a national digital library, they have been talking about this since 1994 (goodness, so much that needs to be cut through to get to where we know we need to get to).

An interesting point from the speaker is that making a metric based on transactional numbers may not be the best way to look at things in a networked world, collaboration is far more important now, than just an impact factor (i cant agree strongly enough with this sentiment).

(Ahh, the famous click stream image is on the screen now (funny that the last time I saw this it was in a talk by David Mccandless, and he was giving out about these ball and stick pictures, of course in the context of this audience, they really have the background understanding to get a lot out of that image, these visualisations have been haunting me for years)

So that was the introduction to the meeting, the keynote speaker is starting now, it’s been given by Charles Henry, the president of CLIR.

Beyond buckets and boxes.

The speaker is going to look at higher education and look at the things that tend to get in the way of realising the vision.

General overview

Ok, these notes are going to be very schematic. One of the problems is that there are thousands of libraries and institutions, and the higher education landscape is one of competition. This world is demarked by solitary comporting entities, the network has no place at all, so these institutions are antithetical to the nature of the networks with which academics deal

Funding patterns, much of huger education has been built project by project, funded projects always have an end, and long term sustainability is hard in this landscape. Yet we know that collaborative grants are going to grow.

Coherence of design

An example is the digitisation of mediaeval manuscripts, there are a number of interesting projects going on, but when they started they didn’t talk to each other. The funding agencies are beginning to understand that they may have been part of the problem over the past 10years.

Another problem is collaboration across disciplines.

Emerging disciplines also present an issue, the Internet brings people together naturally, but it’s hard to sustain that as the people engaged are in traditional departments.

A great example is the hemispheric study of the history of intoxication.

Digital versions do not replicate analog models, and yet we try to make them, and this may be due to the conservative streak in higher education.

Economies, the products of education are public goods, we need to rethink the way we make and save money.

clr mission

Well the challenge is there in front of us, and we need people to do this, the goal must be to rethink the traditional model of what a university and university service level is, by collaborating. Institutions need to become wilfully dependant upon each other.

Discovery, reconstitution, publishing, sharing, augmenting.

The components of knowledge need to be brought into this (there is a bit of a dig at the academic publishing industry).

Presses don’t want to publish unknown scholars, as they don’t sell (agggggg, makes one want to scream). The old model of publication is bankrupt (that is my phrase).

Areas of focus: Coordinated and shared staff, resources, IT.

There are some projects coming up now. West, dpla, 2cul, oapenoaopen, cdl, HathiTrust are some examples.

These projects are efficient, and also disruptive. Disruption can well mean loosing some staff, where some of that expertise is scaled and shared across institutions.

The role of DLF

The DLF is a great place to discuss these ideas, to incubate them, and it may be that this idealised future can’t come to fruition without DLF. Looking at some of the goals of the DLF, they fit ideally within the context of the landscape that has been pointed out.

I got stuck in the Twitter back channel, the closing call for action seems to me to lead to a world that needs quite a lot of structure. If one really wants to take advantage of web scale technologies I think one would need to take risks, but will people be willing to risk their cultural and scholarly artefacts? In the startup scene it’s ok to fail, there will always be another startup coming along, I think that this level of risk does not sit well within the library community, so understanding what we can experiment with, and perhaps just experimenting where we can without thinking about it too much, will tell us more than thinking about it, the university of Huddersfield experiments are a great example of this.

Arriving in Paulo Alto for the DLF fall forum

2010-11-02T00:00:00-07:00

(I’m posting some of these posts a few days after writing, but you’ll get that, won’t you)

It’s five in the morning, and I’m well and truly jet lagged, but after the past week this is not the strangest I’ve been feeling so, it’s not too bad (the immunisation shots for my trip to Mexico left me in an entertainingly woozy state Friday evening).

I’m over to cover the digital library federation meeting, and the satellite SITS meeting on thursday. I’ve been sent along with David Challis from Douthhampton, and allthough we took the same flight we didn’t meet up until we had both made it to the hotel here in paulo alto.

One of the good lessons that came up in our conversation over dinner was that if you running a system that produces a lot of metadata, then it’s really important to be a consumer of your own system, because most of the other consumers of Thea system are not automatically going to alert you when there are errors in your metadata records.

save mailfolder messages to a file using applescript

2010-10-18T00:00:00-07:00

I’ll just start by explaining that I hate applescript. I wanted to create a script that would take a specific folder withing the script and save each message from that folder to a text file on my machine for post-processing.

This is what I came up with.

set fileRoot to "Macintosh HD:Users:path:"
set messageList to {}
tell application "Mail"
	set messageList to messageList & (messages of mailbox "mailbox name" of account "account" of application "Mail")
	set counter to 1
	repeat with theMessage in messageList
		set fileNum to counter as string
		set theFilePath to fileRoot & "mailcontent" & fileNum & ".txt"
		set p to POSIX path of theFilePath
		do shell script "touch " & p
		set theFileReference to open for access theFilePath with write permission
		set messageText to content of theMessage as string
		set sent to date sent of theMessage as string
		write sent to theFileReference starting at eof
		write "\n" to theFileReference starting at eof
		write messageText to theFileReference starting at eof
		close access theFileReference
		set counter to counter + 1
	end repeat
end tell

It breaks down like this, the following script sets a list to be the messages in our names mailbox folder. We can then iterate over that array, getting attributes of each message, such as the message content or the sent date of the message:

set messageList to {}
tell application "Mail"
	set messageList to messageList & (messages of mailbox "mailbox name" of account "account" of application "Mail")
	repeat with theMessage in messageList
		set messageText to content of theMessage as string
		set sent to date sent of theMessage as string
	end repeat
end tell

Taking the creation of the file root out of the loop, and also adding a looping variable out of the loop the content inside the loop now looks like:

set fileNum to counter as string
set theFilePath to fileRoot & "mailcontent" & fileNum & ".txt"
set p to POSIX path of theFilePath
do shell script "touch " & p
set theFileReference to open for access theFilePath with write permission
set messageText to content of theMessage as string
set sent to date sent of theMessage as string
write sent to theFileReference starting at eof
write "\n" to theFileReference starting at eof
write messageText to theFileReference starting at eof
close access theFileReference
set counter to counter + 1

Looking in a bit more detail, we convert the counter (an int) into a string. We set the filepath to a base filename, the string version of the counter, and a file extension. In order to ensure that the file exists we pass this path to a shell script, and “touch” the file. That’s right, in order not to get fucked over by applescripts error handling, I’ve just called out to the system shell as the most useful way of creating a filehandler to a file that might not extist. Jesus.

set fileNum to counter as string
set theFilePath to fileRoot & "mailcontent" & fileNum & ".txt"
set p to POSIX path of theFilePath
do shell script "touch " & p

The second part of the loop now creates the file handler, writes some of the properties of the mail message into the file, closes the file handler, and iterates our loop counter.

set theFileReference to open for access theFilePath with write permission
set messageText to content of theMessage as string
set sent to date sent of theMessage as string
write sent to theFileReference starting at eof
write "\n" to theFileReference starting at eof
write messageText to theFileReference starting at eof
close access theFileReference
set counter to counter + 1

Citation Style Language

2010-09-24T00:00:00-07:00

Martin Fenner asked me to comment on Mendeley’s relationship with the Citation Style Language, so I thought I would pop up my thoughts on the blog too. Martin’s post below is the original message that I sent over to him.

Hi Martin, thanks for getting in touch and asking about Mendeley’s relationship with CSL. There are basically three aspects of citation styling that I’d like to discuss, the first is about Mendeley’s involvement, the second is about the use of citation styling in the community, and amongst publishers, and the third is about how we identify things in academia, and how we format those identifiers.

We have been using the Citations Style Language for quite a while now. We think it is an amazing project and we are very strongly committed to working with the CSL community in encouraging uptake. We get a lot of feedback from our users and one area that they constantly run into problems with is the need to be able to format a citation in just such a manner. The CSL project is the best way for us to be able to support the needs of our users with these kinds of requests. Our developers have been pushing patches upstream to the citeproc-js project, particularly Carles Pina. We have also just added a cut and paste stylebox on our article pages. If you have a look at a sample paper you will now see a little citeproc-js driven box that lets you cut and paste styles in one of the eight most popular citation formats that get used within Mendeley: APA, BibTeX, Cell, Chicago, Harvard, MLA, Nature and Science. We have also been supporting the creation of a WISYWIG citation style editor. The status of the project is that most of the code is complete and we just need to work on getting it integrated into our client, and figuring out the best way to manage the creation of more styles, and how that will work with the CSL community.

One of the things that we have been discussing with Bruce D’Arcus is how to manage the redistribution of new styles, and how to make sure that there corrupt styles don’t propagate, and that people get the style that they are looking for. We have been discussing the creation of a styles commons or public repository, the final plans are not completely in place yet. If people want to contribute there is a lot of activity on the mailing list. One thing we thing we hope Mendeley can help with is reporting usage statistics on specific style files, so at least people can find the most popular version of a CSL file for a given style. I think it would also be awesome if publishers started authoring definitive CSL styles for their journals.

That leads me nicely to talk about the second thing that interests me about citations. After working for many years at Springer, and then Nature, I was well aware that most large publishers just push submitted manuscripts out to companies in India where the formatting of the paper happens. The input format is really for the most part of no importance to the publisher, and also the citation formatting really doesn’t matter to most publishers. They just tear the submitted manuscript to pieces and rebuild it in their chosen XML schema. However, most people using citations are not actually submitting manuscripts for publication, but rather are writing term papers, or theses, or reports. Someone from the Open University recently told me that tutors there can end up docking up to 10% off the marks of a paper if the citations are mis-formatted, and in many cases those tutors have their very own preferred version of a specific citation that they want their students to adhere to.

So the weird thing is that citations started off as a required identifier for the literature. Google scholar and HTTP URI’s have almost totally made formatted citations redundant as identifiers, and yet there is still a huge user need to be able to format citations according to a huge variety of styles, and since that need is going to continue for quite a long time, it’s a need that we have to support.

This kind of leads me to my last point, which is a bit of a lament for the state of informational waste in the academic publishing system. As I pointed out the big publishers don’t care about the submission format, but they have not really done a good job of communicating that to their editorial boards. Smaller publishers don’t have the resources to totally format submissions, and beyond academic publishing there are a huge number of people who just need to format citations. There is a huge waste of people’s time in reformatting papers for submissions, in fixing styles according to changing requirements from departments, and just trying to get the styling correct, when what should matter is the content. I’d love to get to a point where every publisher accepted the same type of XML input, and our authoring tools all created content conforming to that input. Citations should be a HTTP URI that can be rendered into the appropriate format using CSL and and API. Imagine if you had to resubmit a manuscript to a different publisher, and all you had to do was resubmit the same file, just imagine! I’ve been thinking things should work like this since about 2002 when I first started working in the publishing industry.

One of the things I really like about CSL is that is abstracts the form of the citation from the content of the citation. May more and more of our tools do the same!

The Future of Knowledge Organisation on the Web, a one day conference.

2010-09-14T00:00:00-07:00

I’m attending this event today, and I’m going to keep some notes as the day progresses. There are a couple of oddities about the event:

No Coffee. That’s a bit of a pity
No power outlets in the conference room. Not many conferences have that, but one would think that it is a no brainer, especially for any meeting about the web, and in particular linked data.
To add to that, there also seems to be no public wifi. There may well be but we have not been told about it yet.
It also seems that there may be construction happening on the roof later.

On the flip side, the speaker line up looks pretty good, so I’m anticipating learning quite a bit today. There is a huge representation from the BBC. One obviously tends to think of the BBC as an entertainment and news organisation, but at it’s core is an enormous flow of information. There are something like 20 people listed on the attendee sheet from the BBC. I imagine that linked data is pretty important for them. On the flip side, there are very few academic publishers represented, just one as far as I can see, Nature Publishing Group. Their offices are around the corner, and I’ve just bumped into @TonyHammond, who has done a huge amount of work in the past on linking repository systems.

There is a packed room .

So we get going!

Nigel Shadbolt on Government Linked Data, a Tipping Point for the Semantic Web

Wants to talk about policy, the UK is providing a huge opportunity to put Semantic Web to good use. Argues the open government data is a killer sector (in lieu of killer app). Is talking about the importance of local government data, in the new government there remains a clear commitment to transparency in open data.

Interesting point that the scale and scruffy nature of the web has a tendency to catch you out. Hard core AI work often does not scale. The substrate is linked data. (It look likes there is still an attempt in the SemWeb to implement the very bottom of the stack proposed in 2010).

Linked data sets have the same kind of power spectrum as the open way, scale free with a heavy tail. Some resources are very heavily linked to, e.g. some core facts in DBPedia and government boundary data.

As a community we should consider government data as a gift.

The first set of data that prompted opening data was the bicycle accident data. It was opened because someone working in No 10 had a friend who had been killed in a cycling accident. They were astonished when after opening up the data led to apps that had been created by developers. Before this Government didn’t realise the potential in a citizenry that have the tools at their disposal to manipulate that data.

(One of the key points emerging out of this talk is that “interesting data” is the data that has some hook.)

They have created a new Crown Licence that covers data release of government data

They are working towards a definition of what public data is. Basically this is any information that is non-personal and is collected in the course of public service delivery. If this became an entrenched principle then one could imagine finding it easier to hold public services better to account, and could see the creation of innovative apps around that data. (This leads to the question of the power of the citizenry, how does one ensure equality of access to that data, or at least reduce the barriers to the general public for being able to render and work with that data.)

Interestingly he points to NHS dentists, and that’s an app that I used to find my dentist! He also mentions ASBOrometer, an app for looking at the ASBO level of your local area.

He mentions the postcode paper, I was at the Guardian developer day that produced that!! When that developer day happened the Ordinance Survey data was not available, and post codes were a paid for service, but those data sets are now available.

Another example is train and bus time tables. The franchises that run these services don’t have any feeling that information they produce, like the time tables, should be opened. These franchises are running with public money on behalf of the public, and so there is an argument that they should make data of this kind available, rather than just as a paid for iPhone app. An example is Spotlight on Spend which hi-lights government spending.

From January every council will have to publish everything they spend over 500 pounds (I’ll be interested in looking at data from Hackney Council).

One of the strongest arguments for creating “five star” linked data is that it builds a robust national digital infrastructure. If you give a school a robust URI, then every data set that uses that namespace can be reconciled. Enakting is a Southhampton demo of linking disparate data resources together.

Antoine Isaac, SKOS and Linked Data

Antoine is from the Frije Universitat, Amsterdam. He is working in Europeana, and is co-charing the W3C Library Linked Data group. Previously he worked on the W3C group that produced the SKOS specification. His main focus is on cultural assets. SKOS is the Simple Knowledge Organization System. The scope is to represent information around systems such as thesauri and dictionaries in RDF. It is not a language for representing formal semantics, in contrast to OWL. It is possible to turn Knowledge Organisation Systems into ontologies, but it’s kind of hard. They have soft semantics, but that does not mean that they are not interesting. It can be useful for things such as semantic search and annotation systems.

This is my first time in a talk about SKOS. It seems that it underpins something like a thesaurus with terms such as “related”, “broader”, “preferred label” and other terms that give a relationship between terms. These relations are pinned down using RDF and URI’s. This then allows you to bridge different thesauri, as the underpinning relationships in both are underpinned with the SKOS vocabulary. I guess there are tools out there that assist in applying the SKOS vocabulary onto a thesaurus. Interestingly the SKOS vocabulary is extensible, and I wonder whether there has been any move to joig up SKOS and SWAN?

There are some basic constraints in order to try to avoid chaos. One of these is that any concept can only have one preferred label per language. Inference is supported by the concept of “broader” and “narrower” links, and these are inverses of each other, allowing back inference.

There is a minimal semantic commitment, pinning the meaning on the existing information system.

There is an interesting example using library of congress subject headings, which also bring in links to similar concepts from other vocabularies. They stitched together vocabularies from the US, France and Germany.

They have a list of linked data sets that have used SKOS. These include New York Times subject headings and a number of sources from the physical sciences. (I wonder how useful it would be to try to extract interesting terms from scientific articles and link them to New York Times subject headings).

Antoine is starting to run out of time and the convener is standing up and looking leery. Now, time for coffee, yay!

Ricard Wallis from Talis on The Linked Data Journey

“Observations of a fellow traveller”

I recently saw Richard talk at Science Online London, and he gave a great talk there, so I’m looking forward to this.

“Semantic technology has a reputation for being really useful until you add the second user”.

This is very close to the talk that Ricard gave at Science Online, so I’m not going to write up this talk in much detail.

Some interesting lessons coming from the talk, one of the powerful arguments that got government to open up data was to point out to them that the data may well be subject to freedom of information requests. These are very time expensive, and can be totally avoided by publishing the data up front.

When the New York Times published their subject headings they got some vitriolic feedback because of the way that they assigned ownership of heading terms. This was just the result of the naive application of the ontology, but the community got quite upset.

The BBC edit information in Wikipedia for use on their sites. They pull in the data using RDF, and the information goes out into a resource that can be reused by many other people.

Some interesting links:

A site that shows all UK legislation
New York Times data
Animals!
Postcodes!
”Same as” resources
Pivot browser from Microsoft

Steve Dale, Linked Data in Local Government: The Knowledge Hub

So, what is the “Knowledge Hub”? They are trying to solve the problem of the proliferation of sources of information. There are also a proliferation of community web sites. Conversations are becoming more granular, and increasingly dis-aggregated. At the same time there is a growth in data, and data availability. However there is very little connectivity. Wow, he has just pulled up an overview of the proposed architecture. They want to aggregate information from many different sources, add a semantic layer, and create an API an apps store on top of this. They then want to be able to set up a push notification service to users based on context. I don’t believe that this is practically possible at the moment, but if they do build this, then it might well be an interesting innovation.

This general schema is one that could be applied to pushing notifications based on context around many potential contexts, but I don’t believe that we have a good experience for ambient information available to us yet. Aardvark and Foursquare have started to scratch the surface for this, and Tim O’Reilly’s idea for an intelligent phone book are pointing in the direction that we may be heading to, but I think we are very far from that.

This idea seems to me to be lacking in specific measurable gals around success. I notice that they also want to create their app platform on top of the google gadgets platform, so the big question that they have not answered is how to get user and developer engagement. They are starting development now, using an agile methodology they want to get to a live product in early 2011. Good luck to them!

Prof. Dr. Martin Hepp, Linked Data in E-Commerce, The GoodRelations Ontology

From Universität der Bundeswehr München. (Wow, you can really hear the hammering on the roof). In Market economies finding partners for trade is important. This selection process is sometimes done implicitly. The effort for keeping this exchange alive in a market economy is estimated to be more that 50% of GDP in the US. This includes search for suppliers, the banking system, anything that is needed to sustain a market beyond a trivial barter economy. (This seems like a strange analysis, for instance many service industries though probably included in this figure, are also ends in themselves). A key driver for search is specificity of the goods. How much you loose when you can’t use a good for what it was designed.

In 1920 Merck had a catalog of everything that could be traded. There were 5158 items that were relevant for a typical business owner. We are now consuming rather a larger number of types of good. Now in a bakery, for instance, you can get 30 variations of bread! As items proliferate, the search space increases. The effort for advertising, and for searching, for specific items increases with specificity. (I just posted the wrong power cable along with a hard drive, that I recently sold on e-bay!). The web has decreased the search cost, and yet we still spend a lot of time of searches. And yet the web is one of the largest data shredder in history. Often data starts out in a structured way, for instance the producer has a database containing information about the production process. When you transfer this to the web all of this extra data gets stripped.

Linked Data is Data Linked! Preserving the structure of the data on the web, clustering links by meaning, linking data elements represented by documents and reducing the lookup effort for getting hold of metadata for an element. These are the four big things that the linked data world are going to bring into the filed of data management.

Preserving the structure is key, and so the quality of the vocabularies/schemas/ontologies that you use will have an impact on the reusability of the data. Interestingly may shops update data very quickly. Crawling the data does not work if the data is changing hourly, as the cache will be invalid quickly. He makes an interesting point about how a technology will get adopted. It should get adopted when the perceived benefit outweighs the perceived costs. If you have to go out and beg people to use linked data, then there is something wrong with the model.

GoodRelations is a schema for commerce on the web. It has been developer over the past 10 years. You need to realise that an ontology is not just for publishing data, but is also for the structure for reusing the data. It works by getting suppliers to put a payload of RDFa with structured data into a webpage.

When creating an ontology one needs to make it as simple as possible, but no simpler. Getting this balance correct is very important. For instance interestingly in this ontology a price is not a property of a product, but rather of an offer. Stores and business entities are distinct. A product is distinct from a product model.

The basic structure is that there is an Agent who makes a Promise around an Object.

His closing point about making use of RDFa over SPARQl endpoints is brilliant. As he says, connecting 80 databases using a line call is something any middleware vendor could have done in the 1980’s, how do you convince thousands of shop owners to do this? The answer is that you don’t.

Andy Powell, Eduserv, Linked Data - the long and winding road.

This is a somewhat more sceptical approach, not totally sceptical. If we believe that linked data is the future of the web, then what we are really saying is that RDF is the future of the web. Looking at one community we can perhaps learn some lessons, let’s look at the Dublin Core community. The key-points of this talk are:

DC is originally 15 metadata elements
in actual fact its more like 60 properties and classes, that is maintained reasonably well
this is all declared using RDF/RDFS
it started in 1995, a very librarian-centric view, thinking one could still catalog all of the web pages
- embedded in meta tags
- expose this to search engines
One of the consequences is that there is a record-centric approach
so that you can ship them from one application to another
the record is the mechanism for tracking provenance
the original elements were thought to be 15 fuzzy buckets * this was a feature, not a bug
the data modelling was very naive, e.g. author name was an item in a DC record rather than thinking about it being a property of a person who was the author
strings vs. things, a separation between entities, and the labels of those entities was not properly resolved
little abstraction of the model from the syntax of the underlying model *
Challenges
Even still a discussion about adopting an RDF model
need to argue towards Open, in that you don’t want to be in a silo
you need to recognise that everyone can catalog stuff, and for the more traditional cataloging community this can be hard to accept
the ‘http’ URI problem. * people stil think that http URI’s are only for locating web pages, need to conceptually get across that you can use them to identify anything
modelling * modelling is hard, they also need to gain traction within a community
the modeller is a great post highlighting some of the issues
linked data must be a success on the web if it is to play a future on the web, that means RDF needs to be a success on the web
so far it has not been a success on the web, which is not to say that it won’t in time.

So far this has been the best talk of the day, it’s really great as an overview.

John Goodwin from Ordinance Survey, Linking to Geographic Data

Research Scientist with OS. At OS they are very good at saying where stuff is, but not what stuff is. Geo data is important, there are still confusing records around place names, e.g. Hampshire. The OS has decided to open an authoritative set of linked data, e.g.

boundary-line esri-shape file
code-point
gazetteer

How do you get that out as linked data when the formats are complex and sometime proprietary?

They have now converted this into linked data. RDF is not very good for spatial data, but you can do some qualitative relations, and you can put in topological relations, such as containment, and boundary relations. This administrative info is on the web now. (The postcode paper gets another mention, yay, go PostCodePaper!!)

Andreas Blaumer, PoolParty:SKOS Thesaurus Management utilising Linked Data

from punkt.net Services.

This talk starts off by re-iterating the power of the web. It looks like many apps are beginning to sit on top of web scale information, and not being based on heay ontologies. As Jim Hendler says “A little semantic goes a long way”. For a system that seems to be being based on the topology of a large scale network I’m surprised that no-one has been talking about the implications or details of the structure of the network. No power spectra in this conference yet. It was alluded to a little by the person who was talking about GoodShopping.

Andreas thinks that SKOS could introduce Web2.0 mechanisms to the Web of Data. When they developed Pool Party, then the question is how can any person publish information or knowledge, as linked data? (sounds a bit like the Drupal extension).

The tool seems to be a web interface that allows for markup and concept tagging of documents. Can connect to a corporate thesaurus. Wow, we are going to get a live demo.

They start by taking a text and counting terms. Then they look for terms from DBPedia, as well as calls to a thesaurus for economics. They also look for related terms based on completing trigrams, and use these to generate a SPARQL query to get even extra terms. These terms are then presented as a set of possible tags that can be used to semantically mark up the text. They have a cross-language dictionary that helps cross-tag across languages. They have exact and close match in searching. They don’t use “sameas”. You can also do a matching between concept schemas. They can import a number of different kinds of info, N3, N-Triples, RDF etc. Output is possible in a number of different XML formats.

He mentions the LOD2 project from the EU. This is a 4-year EU project. Good luck with that. This is an FP7 project. The EU is making a lot of data available, perhaps there will be scientifically interesting data sets being made available.

Bertrand Vatant, Porting terminologies to the Semantic Web

a.k.a. the Semiotic Web (you can tell that the speaker is French, I think I have only ever heard the word Semiotic being used by French people). On the slide there is a reference to the “quick off” of datalift. (he means “kick off”, but it’s a pretty cute error).

He has just said that a resource as in ‘URI’ has become more abstract over the last 20 years. It is now a resource if it is identifiable and worth speaking about (mind you most things that are addressable are really not worth talking about, certainly not most things that are on the web). He really is talking about semiotics and semiotic triangles. And this is the last talk of the day. (The open web, in a way, wallows in it’s own existence without too much care for the signifiers. It’s a dynamical process, an infrastructure, along which meaning travels. Stopping to pin URI’s on the evolving mechanisms of the web may not be a viable strategy. The implication for this is that systems such as twitter will never emerge on the semantic web as the semantic web will not be able to evolve at the scale that a system like twitter can evolve. That’s not a bad thing, it just means that we need to be clear about the potential for these kinds of tools, and to be aware of the correct domains where they should be applied.)

And at this point it’s time for a glass of wine.

links mentioned in this post

Slides from APS Talk, August, 2010

2010-09-13T00:00:00-07:00

Back at the beginning of August I gave a talk at the American Phytopathological Society. It was a great opportunity to talk to some really interesting scientists. My talk slides are below.

Unveiling the web, making the implicit explicit.

View more documents from Ian Mulvany.

Connecting Scientific Resources, Slides

2010-09-07T00:00:00-07:00

On Friday I hosted a session at Science Online London. Michael Habibi, Richard Wallis and Chris Taylor all gave great presentations. Michael’s presentation and Richards’s presentation are both online, and when the organisers post the other talks I’ll link to Chris’s presentation too.

Michael, taking us through his talk

Richard has a rapt audience

Chris, getting passionate

science scraping with YQL

2010-09-06T00:00:00-07:00

Last Saturday at Science Online London I gave a quick tutorial on YQL, and how it might be used to mash up scientific data sets. Below I list some of the sample queries that I was playing with. Before you get started with the console have a look through the documentation. I got a lot of milage out of the part about filters and joins. The blog post by Paul Hogan on using YQL for library maships was also very helpful.

In my presentation I was originally looking at extracing data from a report on soy bean rust spread.

Here are a few sample queries to get you started

select * from html where 
url="http://sbr.ipmpipe.org/cgi-bin/sbr/county_info.cgi?date=2010-07-13&pest=soybean_rust&host=All%20Legumes/Kudzu"

This query just pulls all of the HML from the page. Open in console.

select * from html where 
url="http://sbr.ipmpipe.org/cgi-bin/sbr/county_info.cgi?date=2010-07-13&pest=soybean_rust&host=All%20Legumes/Kudzu" 
and xpath='//table'

This query extracts only the table element from the page. Open in console.

select * from html where 
url="http://sbr.ipmpipe.org/cgi-bin/sbr/county_info.cgi?date=2010-07-13&pest=soybean_rust&host=All%20Legumes/Kudzu" 
and xpath='//table/tr/td[2]'

This query pulls the second item from each row of the table. Open in console.

select * from html where 
url="http://www.mulvany.net/files/ipmsemanticpipe.html" 
and xpath='//table/tr/td[@id="status" and p="Confirmed"]/..'

For this query I copied the table onto my own server and added some basic proto-semantic markup to the column descriptors. I could then call out specific columns from the table. Open in console.

select * from csv WHERE 
url="http://www.mulvany.net/files/ipmpipe.csv" 
and columns='date,place,status'
and status='Confirmed'

With this query I converted the table into a csv file. This demonstrates YQL’s ability to query against csv files. Open in console.

Connecting Scientific Resources

2010-08-27T00:00:00-07:00

I’m going to be hosting a session at science online London next weekend, I’m excited. I’ve been interested in the issues of connecting scientific data for a long time. In the last six months I’ve become particularly excited about the potential of web based tool like Yahoo Query Language. I was hoping to talk a little about that, but I’ve been lucky to get some amazing people to come and share their experiences about linking data, so I’m going to cede the floor to them. I might be able to get some YQL hackery into one of the unconference slots that will be knocking around. Science online is shaping up to be a pretty awesome event, and you can check out the conference program to see what you will be missing out on!

Here is the spiel and speaker bios for the section that I’m going to be running:

Connecting Scientific Resources

Do you have data? Have you decided that you want to publish that data in a friendly way? Then this session is for you. Allowing your data to be linked to other data sets is an obvious way to make your data more useful, and to contribute back to the data community that you are a part of, but the mechanics of how you do that is not always so clear cut. This session will discuss just that. With experts from the publishing world, the liked data community, and scientific data services, this is a unique opportunity to get an insight into how to create linked scientific data, and what you can do with it once you have created it.

About the Panel

Ian Mulvany, VP New Product Development, Mendeley

Ian Mulvany is VP of New Product Development for Mendeley.com where he is responsible for ensuring that the tools being built really respond to the needs of researchers and scientists. He spends much of his time examining the implications of emerging web technologies for the practice and communication of science, and he is excited by the untapped potential in linking and exposing data.

Michael Habib, Product Manager, Scopus UX + Workflow

His specialty and passion is designing interactive and social experiences around content and metadata. Michael is currently focused on improving the Scopus user-experience by pursuing tight and seamless integration of Scopus and Scopus data (APIs, etc.) into the STM literature research workflow.

Richard Wallis, Technology Evangelist Talis

As Technology Evangelist at Talis, he is at the forefront in promoting, explaining, and applying new and emerging web, semantic web and linked data technologies. Richard is a well known speaker at conferences and events, providing entertaining and informative insights in to Web 2.0 and the Semantic Web and their influence upon real world situations.

Chris Taylor, Senior Software Engineer for Proteomics Service, EBI

As Senior Software Engineer, Chris plays a key role in the The Proteomics Services Team, providing databases and tools for the deposition, distribution and analysis of proteomics and proteomics-related data. The team contribute to the development of community standards for proteomics data in the context of the HUPO Proteomics Standards Initiative (PSI), and develop reference implementations for these standards. Chris is also involved with the isainfrastructre project, a tool to assist in the annotation and local management of experimental metadata.

Passing arguments in liquid templates.

2010-08-23T00:00:00-07:00

I’ve recently moved my blog over to hosting on github using the Jekyll templating system. I was intrigued by the ability of Jekyll to create a listing for related posts, but in the end the quality was poor. There are two methods avilable, the more robust uses latent semantic indexing, but this is not supported by github. I can understand this, as the processing required is a bit too much of an overhead for a lightweight hosting service. The other method is a fast method, but for all of the posts that I tried it against it only ever produced the last three posts that I had created.

I really liked the idea of having some related content after each post, so I decided to pull out posts that were related by tags. I like tags a lot. In order to do this I needed to iterate over each set of posts for each category from the categories for the given page. This looks something like:

{% highlight html %} 
{% for category in page.categories %}
  do stuff
{% endfor %}
{% endhighlight %}

The problem that I was having is trying to figure out how to pass the value of ‘category’ to the next loop. I didn’t find any documentation, but after trying out some random syntax hit on the following, which seems to work:

{% for category in page.categories %}
  {% for post in site.categories.[category] %}
	do stuff
  {% endfor %}
{% endfor %}

The important line is: {% for post in site.categories.[category] %}

And the piece of syntax that make is work is encasing category in brackets [category].

Testing Code Hilighting

2010-08-15T00:00:00-07:00

testing ruby

def foo
  puts 'foo'
end

testing python

for nice in this:
    print "how interesting"

testing latex

\[ 
\sum_{n=1}^\infty \frac{1}{n} 
\text{ is divergent, but } 
\lim_{n \to \infty} \sum_{i=1}^n \frac{1}{i} - \ln n \text{exists.} 
\]

Notes from JISC activity streams workshop

2010-07-14T00:00:00-07:00

Dave Jennings giving the opening talk, http://www.slideshare.net/davidjennings

abundance of content without structure is not to be feared, example comes from music. uses a Medneley slide to talk about the lastfm – Medneley bridge. talks about the amazon collaborative filter mentions how this can lead to a dystopia, Googlezon, the big brother of recommender system. talks about the future of pervasive recommender systems, and brain matching to content matching. john cage - my favourite music the music I haven’t heard yet.

search browsing monitoring wait for interesting things to pass by you

there is a social aspect, you see the tracks that others have left there is a bandwagon effect (the network clustering effect) black sheep element - people like to be individuals

Another dystopia, tower of babylon, how do you organise this? (take a leaf out of the anarchists book)

flickr does this with the use of interestingness (I like his description of this as a first part in a dialog)

Jenning’s law “people make most of their discoveries elsewhere” (serendipitous, algorithmic recommendation, word of mouth)

Feyerabend - the conquest of abundance

In what way are the organisation represented in this room different from more commercial organisations?

The academic operation is not framed around a totally production oriented model, and this is one big way in which these are different.

Dave Pattern University of Huddresfield,

These guys are using usage data.

a path though the history of information in supermarkets:

history of user data in retail

stamps with prices
barcode
stock control
then there were loyalty cards for data mining across transaction history - customer profiling (will libraries ever do this kind of customer profiling, at the moment they don’t)

what they do at huddersfield

22k students
2k staff
300k books
3M borrowing records

not much had been done with this data, they stared playing around with this data

people who borrowed this borrowed that
initially they didn’t know if this would be useful a librarian dissed this as something that amazon does to sell books, not the kind of thing that a libarary should do.
logged in giving users a history of what they borrowed recently
show lending paths, which books got borrowed with books (that’s brilliant,)

they were also able to create new book lists for the entire library (lists of new books coming in to the library)

they profiled the borrowing of students from a course, based on dewey numbers. as new books with those dewey numbers com in to the library, these get pushed to that class (also genius)

on the front page of library catalog, they pushed in the most popular keywords onto the front page as a tag cloud. this seeded a spike at the start of the year on those keywords for new searches

they can also push related keywords along with keyword search this came up in the meeting on our new universal search bar

they have clickstream data, but they don’t know what to do with this yet.

they know what they most popular keywords are that lead to a specific book, but they don’t know what to do with this data yet.

measuring the impact

from 2005 their range of unique titles that are being borrowed go up from 65k to 80k in 2009, trend continues to rise, this comes from the layer of serendipity

it’s not clear how much of this increase comes from these new tools, but certainly it’s not driving to a homologous self similar borrowing behaviour

also average number of books per student is going up how was that number calculated, as a student average, or based on per student numbers

more book-loans correlate with better grades <- this is very interesting.

sharing data

giving the data to students in a BA design course, and creating interesting visualisations

at the end of 2008 they released book circulation recommendation data. it’s important to attach a licence with the data, they are using an opendata licence, public domain,

within a couple of days someone created a semantic representation of the data.

“the coolest thing to do with your data will be thought of by someone else” - Rufus Pollock.

summary

QA

Q what kind of conversations have you had with your academics A the attitude was less why should we, and more why shouldn’t we, academic feedback has been positive

Q from paul walk, data needs to be managed because raw low quality data is not useful

Q Richard Geddis, OUP, Counter, and something else would it be less easy to do recommendation for journal articles than for books, talking about

[[ how many books with dewey codes are represented in our catalog? ]]

There’s something going on, title of the first debate

ken chad

point is “how can libs make better us of the data that they have, great rich bibliographic data”.

it’s not an ‘it’, the data is valuable, is libs don’t use this, then perhaps someone else will (facebook).

paul millar

have been concentrating on measuring activity in the systems that libs have, but perhaps been missing the context. as we get a point where we can do something interesting with activity data, perhaps we won’t need it as we will have other ways of getting recommendations.

with social networks people are becoming the entry points into … ? what? how do we accommodate social networks into libs approach going forward

business intelligence needs context to be useful. context is not easily derived from a single system there needs to be ways of adding context from different systems perhaps one should mention the work being done by Ciro’s group??

user should have control, the ability to add value to a system he mentions that there are risks involved, anonymity can be reverse engineered (reminds me of paper on network anonymity)

who should own the attention data? mention pubsubhubbub an open licence a trusted party the government google?

is afraid that we will go through a centralised service, like facebook, initially

richard nurse

approach it from a business intel point of view, rather that the user data.

is it in the best interest of the institution?

essential message is that institutions need to be more pro-active.

four key reasons for why institutions should get involved

diversity of users
direction though data, navigation through that data
degree of control
duty of service, because of public investment

there are risks, privacy, legal etc, but these also depend on institutional policy

need selectivity on which users institutions address

if institutions don’t build an understanding on this information, then others may step in and create services in their stead (could end up working in partnership, but at least institutions need to have a seat at that table.)

Q&A session for this debate:

Joy Patten from MIMAS do institutions have the level of data that huddersfield have some have and some don’t (TILES project) then at what point do you need to look at a national strategy what do we know of the goldmine of data? (that seems like a silly point)

A: informed by the maturity of systems
	(I would say that it has to related to the prevalence and openness of identifiers)
Paul Millar opposed the idea of aggregating this kind of data.
is afraid of having the data cornered by commercial institutions
this leads to him wanting to see the data centralised and control

[[ send notes to David Kay ]]

debate 2, love data, hate silos

open university case study

most students are not on campus, they are not registered with the LMS

Richard Nurse leads a discussion about the kind of systems that students use, leads a discussion about what kind of information students might interact with, listed on whiteboard take a photo of this

there are a huge number of sources for data
	e.g. 
	reading lists
	borrowing systems
	site access
	access control systems
	VLE data

	uk borders agency have a requirement to know what points of contact people  on student visas have with the institute? (need 10 per year)

an interesting question is how accessible is this data
	people have a day job, and getting the data out requires time

someone asks "why are libraries collecting this data?"
if it is used for business intel for institutions
  
Q: do these data exist in a way that can be pulled together?
	- yes, but there is a ?? on showing that one can be trusted with this data
	Manchester Metropolitan have created uniview (a students record driven data set, they are now starting to mash that up with VLE click data, and that is giving interesting information on success, 
	e.g. don't fiddle with the VLE area
	if staff put too much information into a VLE there is a negative impact on student performance

	context is everything for interpreting this data

there is a lot of evidence that students prefer to interact with others and teachers through 3rd spaces, social networking spaces, such as facebook.

students like groups, but they don’t want the institutional activity to appear directly on their spaces on facebook

interesting questions: would any of these people consider opening up some more of this data as “open data” (somewhat of a stunned silence)

interesting point, will government enthusiasm for open data affect universities - teaching quality information should be made available, for instance

OU have started thinking about mining this, using SFX for instance one of the big problems is getting access to that data, the structures that manage things like the VLE are not directly involved in doing this investigation, so you need to get high enough up on their agenda.

one could go an look at user generated content, book reviews from amazon, for instance.

have started to use google apps, are thinking about building custom apps for their users

my comment - identifiers are key

other comment - data warehouses are hugely valuable. comment: these are mainly being built at the moment for institutions to make institutional decisions

not currently being thought of as being used to drive student services, but one could get that in via the back door.

debate 4, appropriate or inappropriate use, legal issues

privacy is not the issue, control is

some opening remarks,

the landscape of the data is not simple, all of these cases will apply, open/closed legal/illegal appropriate/inappropriate

often some of this is related to catalog entries, who creates the catalog entry.

all of the data under discussion is currently under the control of institutions.

there are clear legal provisions 17 yo students are children adds requirements to what you can do with data

privacy

access

comment, legal requirements can change. mentions the downstream dilemma,

naomi klein – works on IP, and has looked at IP and bibliographic information.

mentioned that the information commissioner has issued a guidance that you should treat any such information as personal information

jisc legal mentioned some guidelines

need to keep personal data secure
there is a danger of triangulation

question what are we risking by not doing something with this data?

a: we risk missing opportunities to cross reference data, and to create new connections.

conversation reverts to legal FUD discussions

one risk in not doing anything is getting labelled that ones institution is not innovating, and hence not providing value for money.

feedback from two evening debates

questions about evidence and whether we should “just do it”

Elib was the 1000 flowers bloom approach

there is a fear here that funding is going to be reduced, so not doing this is going to be a road to ruin,

doing these things is going to be a need to do thing, in order to enhance the learning experience.

increase student satisfaction
increase student progression

this means service delivery, and learning enabling need to focus on tools (?) that can drive satisfaction and progression.

afternoon session

interm report on mosaic

they are looking to build a version built on lucene, hadoop and linked data.

mark@headtech.com

mark toole (director of stirling information services), richard korn OU, naomi klein legal

Mark Toole

some benefits is a fascinating area could see some things that could be made

some costs biggest cost is priority, working on this stops him working on other things. how do the benefits from this contribute more than resources put into other efforts, buying more books, for instance.

in the end, there is not enough quantifiable evidence, we need more real case studies.

comment: challenge the idea that there is not enough evidence. 
	there is a large body of existing data, e.g. known algorithms, amazon, understanding of privacy issues. it might be outside of the sector at the moment, perhaps we need within sector examples [[ our expert finder could be a good example within this context ]]

richard korn, open university

nice quotes from wall-mart, google, and ms, data should be used to change the services that produce the data, not simply seen as a by-product of the data.

mentions SFX, are thinking of playing around with an api to improve this.

nice example from search results. 

Telstar is using SFX to link to resources

linked data for OU content, Lucero

slideshare.net/richardnurse

naomi klein, getting business intelligence from user activity data, legal challenges

areas of law that touch on this

DP act 1998
human rights act
right of privacy
database rights
contract law

with minors there are further issues

there may be IP rights associated with a ‘mere fact’ if it is collated in a DB, or if it has been enhanced through a process, e.g. recommended

built a tool hosted by jisc legal that gives info on IP rights on bibliographic data

the data may be provided under contract from a 3rd party, you may be bound by those contractual obligations.

Data Pretension act in a nutshell: if the individual can be identified it is classed as personal data

there are a a few debates about this issue

if it is personal data, you need consent

JISC legal is publishing a report on consent management

institutions may have a get out clause, regarding requirement to provide services (was that right?)

look at web2rights toolkit

mentions cultural fear in the face of data protection issues, very good point, one should not be frozen into inaction, with the appropriate information it is possible to navigate these issues.

when might activity data be personal data?

- cloud computing and personal data
	if the information is processed externally, and can be tracked back to an individual

- use of social networking

- google and usage data, e.g. search data.
	triangulation

- library activity data

recommendation - any info that provides an indication of a users’ online activity should be treated as peronal data, even if individual can\t be identified

role of passing on user data to 3rd bpodies

the role of the institution:

bottom line need to ensure that you have consent before processing personal data

anonymous data still needs to be checked against risk of triangulation.

evening session making recommendations for JISC activity on a national level,

we need to give some concrete suggestions on a national level:

davep produced data, and the world didn’t end data model from jisc mosaic project there is potential for legal advice

is next step then is to encourage the creation of more services?? clarify what we have

thinking nationally, what is the case?

can we give more compelling business cases, mosaic project ran into trouble with senior stakeholder buy-in

again the connection between item usage and performance indicators is hi-lighted as being hugely important.

the performance data is contained in something like a VLE,

HESA data provides something in the public record, though it may be in a somewhat convoluted form.

many systems are not even collecting data at the moment, they sort of need to collect the data if they want to use it at some point.

recommendations

national infrastructure

find a way to test the hypothesis that extending the data towards a national level could provide value
get institutions to start tracking this data, for if they don’t they can’t use the data in the future, should they wish to

learning & research recommendations

cookbooks for VLES, how do you get activity data out of these vendor systems
who are the stakeholders that need to be won over? involve them, they may have some experience to lend to the issue
researchers are a problem, they go about community building in a totally different way, they may be looking to things like Medneley to get information about research communities, rather than something like the VLE

library and local collections

concern of duplication of effort, need and a desire to share knowledge, so for example methods for getting data out of systems can be shared
need some case studies that could support the anecdotal stories
concern that we don’t know what data we are going to share, address this with a survey to see what’s been done, and what can be done in library systems.
who is the authority on this, if it’s not dave, who is it?
something about this and books, and numbers of books, missed that bit.
discussions around reading lists, and how useful students find these to be, some lecturers have IP around their reading lists, making this difficult to share.

specific recommendations:

need to profile student behaviour in terms of e-resources, not just books
need strong evidence that library services are integral to learning, and not just an addon.
open source systems
if we share our data others will find the interesting things within the data.

end comments

this community needs to be accountable, usage stats and services built around that can be one way to do that

Read mail in intervals on OS X

2010-07-05T00:00:00-07:00

In order to check mail less frequently than the options available, following a hint from macosxhints:

set up an applescript to tell mail to check for mail.
put that script under a launchd deamon with an interval
use lingon to setup the deamon

so far it seems to be working, yay!

http://www.macosxhints.com/article.php?story=20010427022452992

use

my first post

2010-07-03T00:00:00-07:00

I’m trying to understand jekyll here

It seems to be quite nice, let’s see how we get on!

Paging through results on google app engine

2010-06-04T00:00:00-07:00

The sample code given on the appengine article about paging through results is incomplete for the method of paging through a results set by item key. Here is an example that will page forward and backward through a result set by item key. This example will display items oldest first. The next link will bring you to the set of next newest items.

template code for displaying the results, let's say it's called listitems.html

the relevant code in your application file.

 
#assume you have imported all of the googley goodness

#this sets the paging size
FETCHLIMIT = 10

# an empty class for testing against.
class Item(db.Model):
    created = db.DateTimeProperty(auto_now_add=True)

class ListItems(webapp.RequestHandler):
    def get(self):
        query = Item.all()
        items = query.fetch(FETCHLIMIT+1)
        
        forward = self.request.get("next")
        back = self.request.get("previous")

        if forward:
            items = Item.all().order("__key__").filter('__key__ >=', db.Key(forward)).fetch(FETCHLIMIT+1)
        elif back:
            items = Item.all().order("-__key__").filter('__key__ <=', db.Key(back)).fetch(FETCHLIMIT+1)
            items.reverse()
            # in order to go backwards through the set we need to revers the order so that we get the newest keys on top
            # the filter than drops the items we have already seen
            # but to get the previous link to pass the oldest key in the set we need to reverse the items before heading on
        else:
            items = Item.all().order("__key__").fetch(FETCHLIMIT+1)     
       # the key part here missing from the example code is making the right call against the right passed variable
       # forward = self.request.get("next") returns a string, we convert that to a key object with the line:
       # db.Key(forward)
             
        # do we show next and previous links?
        # start with none
        next = None
        previous = None 
        # we need to get the key of the very first item in the results set, 
        query = Item.all()
        first_item_key = query.fetch(1)[0].key()
        previous = activities[0].key()
        # if previous key is the same as the key of the first item in the class then don't display a previous link
        if previous == first_item_key: previous = None 

        # we have just asked for a batch of items. if the returned set is bigger than the amount we want on a page
        # we know there are some results left to get so we set a next link
        if len(items) == FETCHLIMIT+1:
            next = items[-1].key()
            items = items[:FETCHLIMIT]
        
        # perhaps we want to display the total count of objects in the datastore
        item_number = Item.all().count()
        template_values = {'count':item_number, 'items':items, 'next':next, 'previous':previous}
        path = os.path.join(os.path.dirname(__file__), 'listitems.html')
        self.response.out.write(template.render(path, template_values))

Paging through results, google appengine, newest first

2010-06-04T00:00:00-07:00

Modifying the Listitems class from the previous example

 


class ListItems(webapp.RequestHandler):
    def get(self):
        query = Item.all()
        query.order("-__key__") #chronological reverse order
        items = query.fetch(FETCHLIMIT+1)
        
        forward = self.request.get("next")
        back = self.request.get("previous")

        if forward:
            items = Item.all().order("-__key__").filter('__key__ <=', db.Key(forward)).fetch(FETCHLIMIT+1)
        elif back:
            items = Item.all().order("__key__").filter('__key__ >=', db.Key(back)).fetch(FETCHLIMIT+1)
            items.reverse()
            # in order to go backwards through the set we need to revers the order so that we get the newest keys on top
            # the filter than drops the items we have already seen
            # but to get the previous link to pass the oldest key in the set we need to reverse the items before heading on
        else:
            items = Item.all().order("-__key__").fetch(FETCHLIMIT+1)     
       # the key part here missing from the example code is making the right call against the right passed variable
       # forward = self.request.get("next") returns a string, we convert that to a key object with the line:
       # db.Key(forward)
             
        # do we show next and previous links?
        # start with none
        next = None
        previous = None 
        # we need to get the key of the newest item in the results set, 
        query = Item.all()
        query.order("-__key__") #chronological reverse order
        first_item_key = query.fetch(1)[0].key()
        previous = activities[0].key()
        # if previous key is the same as the key of the first item in the class then don't display a previous link
        if previous == first_item_key: previous = None 

        # we have just asked for a batch of items. if the returned set is bigger than the amount we want on a page
        # we know there are some results left to get so we set a next link
        if len(items) == FETCHLIMIT+1:
            next = items[-1].key()
            items = items[:FETCHLIMIT]
        
        # perhaps we want to display the total count of objects in the datastore
        item_number = Item.all().count()
        template_values = {'count':item_number, 'items':items, 'next':next, 'previous':previous}
        path = os.path.join(os.path.dirname(__file__), 'listitems.html')
        self.response.out.write(template.render(path, template_values))

hackney are fuckers.

2010-05-12T00:00:00-07:00

http://www.whatdotheyknow.com/request/31711/response/79454/attach/html/3/1%202181984%2020100407%204th%20Tier%20List.xls.html

fast food in hackney
http://www.whatdotheyknow.com/request/27794/response/68154/attach/html/3/1%201952373%20Information%20Sheet.xls.html

Edinburgh, first draft

2010-05-04T00:00:00-07:00

I don't post my poetry often, indeed I don't write often, but I thought this one came out OK. Written over the past weekend for a good friend on the occasion of her birthday.


Edinburgh, first draft, 01/05/2010

for andrea.

Was it in Summer, when I lost myself among the graveyards?
The moss covered steps of Greyfriars green and gleaming.

Or was it someone's winter night along cobbled streets, 
where buildings wear a sharp puritan cut to their overcoats?
Hightailing it though the cold from one warm pub to the next.

Could it have been an Edinburgh Spring?
Beltaine roared along High Street in the gloaming
The burning processing smoking up the back of Calton Hill
A firebreather bellowed,
Gouts of flame framed faces.

If not in these times, 
then it must have been an Autumn loamy wet and windswept afternoon.
Sitting in the back of an Elephant,
last light out of the west washing against the castle.

It threaded itself, wound round us,
as we wandered Whitehouse Loan,
High Riggs, West Port, and up the Vennel Steps. 

In such moments that city passed beyond us being there,
 and into our dreams.

A mini rant about LaTeX

2010-04-29T00:00:00-07:00

Here is a comment I've just posted to a comment thread about LaTeX over here: http://blogs.nature.com/farhat/2010/04/16/collaborative-editing-with-latex. One of the commenters asked "Why not just use a word processor"

Just using a word processor simply does not cut it when it comes to expressing complex mathematical ideas. LaTeX, and it's precursor TeX, have evolved to be the only tools that can currently express the richness of the ideas within advanced mathematics. Whether this is a good thing or not is moot, it's simply the case, so that's one reason to use LaTeX.

A huge advantage of using a text only tool is keeping different versions of a document, and diffing between versions, is much easier than the horror that is "Track Changes" on most editors. Many groups I know keep their papers as LaTaX files in a code repository and use a system such as SVN for collaboration. This is not a benefit of LaTeX specifically, and is way beyond what most people would do with it, however the need for this kind of ability with the objects upon which we collaborate is painfully clear, and is only starting to be address in the "rich document" space with services such as Google Wave and Google Docs.

Ideally one would like to be able to write one's equations directly into the computer. For the time being LaTeX provides an expressive syntax from the keyboard that translates into mathematics on the screen. The iPad may well change this, but I suspect that any solution that takes handwriting and converts it to marked up mathematics will be built on top of the excellent codebase that is the LaTeX framework. This would be an excellent example of the DRY formalism (don't repeat yourself), build it on top of that which already works. For an example of a web app that does this already have a look at:
http://detexify.kirelabs.org/classify.html.

Richard is right that not separating the semantics from the presentation is a weakness. Sadly one issue is that a single equation or symbol can actually mean different things. The closest tool there is to encapsulate the semantics of mathematics is the semantic version of MathML. No one writes this by hand. The best tools for producing semantic MathML (as opposed to the presentation version of MathML), do so by translating from human input in the form of, yes you guessed it, LaTeX.

In spite of being semantically dumb there are a couple of interesting web services that build on top of the fact that there is a huge community of people out there who speak LaTeX. http://www.mathtran.org/formulas/ allows people to share formulas. http://www.latexsearch.com/ allows you to search through Springer's archive of mathematics. Springer has the most extensive archive of mathematical literature in the world, so that is no mean feat.

The last thing I will say is that LaTeX is just awesome. It produces beautiful documents. It is akin to learning a programming language, but then, if you are doing serious mathematics, it's just another symbol set, it's not that hard, really. I guess if you are doing something soft and not so well defined, then a word processor is going to be good enough for you.

creating a tumblog

2010-04-13T00:00:00-07:00

After a few conversations with @zzgavin, I've come to the conclusion
that there is a place for a tumblog in the toolkit of those who like
to post content onto the web. There are often pieces of content that I
want to call out, but don't have either the time or the interest to
write a longer blog post about. For a while I thought that would be a
good use of Twitter, but in the end that type of behaviour just
pollutes your twitter feed. Indeed, posting a link to an image is
never as immediate as posting the image itself.

This blog will remain a repository when I want to say something
coherent that is about 140 characters. My twitter feed will continue
to respond to the question of what it is that I am doing, combined
with it's use as a lightweight communication network. My tumblelog
will henceforth be an online digital clipboard of links, quotes and
images that tickle my fancy.

If you are so interested you may check it out at:

http://attention-deficit.tumblr.com/

The three ages of Elvis in potatoes!

2010-03-22T00:00:00-07:00

potato_3stagesElvis 004
Originally uploaded by gioia_

On Saturday we went to a party thrown by some friends of ours. It was kind of a delayed St. Patrick's day party, and everyone had to bring along a potato dressed up as a celebrity, or some such.

We created our potato after the Father Ted episode "Competition Time", in which Ted, Dougal and Jack dress up as the three ages of Elvis. The third age in this little diorama is made up from mashed spud in a sock.

After an intense voting round, we were finally pushed out of 3rd place by St Patrick. The winner was Spudward, which looked disconcertingly like the pop group with it represented.

Kierkegaard, forgetting and existence

2010-03-17T00:00:00-07:00

Reading though the excellent introduction to Kierkegaard's philosophy
in the Guardian
(http://www.guardian.co.uk/commentisfree/belief/2010/mar/15/kierkegaard-philosophy-existentialism),
it is pointed out that one of the key underlying principles of his
philosophy is that we have forgotten what it means to exist, and that
the cause of this forgetting is the ever growing amount of knowledge
in the world (a state of affairs that is now ever more apparent even
than in his time).

I am dubious about this claim. I think it likely that we have always
been forgetting this questions. In the trials of agrarian society the
question is pushed aside by concerns of the harvest. In the hold of a
religious society such questions are givens and not open to
inspection. It's always a hard question, and I'm sure that modernism
causes it's disappearance more than any other state. Of course I
could be wrong.

Python PEP for a graph API

2010-02-19T00:00:00-08:00

I just stumbled across this
http://wiki.python.org/moin/PythonGraphApi. I think it's great. The
discussion has actually been going on since Aug 2004, so I don't know
what the status of this PEP is. I would love to see something come out
of it eventually.

tags: graph, PEP, python

Probabilistic language models, auto-correction tools and scientific discovery.

2010-02-10T00:00:00-08:00

Probabilistic language models, auto-correction tools and scientific discovery.

"Durgesh Kumar Dwivedi":http://network.nature.com/people/U56CB3E51/profile over on Nature Network just asked "Does anyone have any software or web address which corrects English grammar, preposition, edit and shortened the paragraphs?". This question brought to mind and idea that I had a few years ago.

The idea is simple enough, use a large corpus of pre-vetted grammatically correct text as a training tool to compare sentences against. If you have enough example sentences, then every occurrence of every word in a given sentence will have a certain likelihood of occurring. Errors, and new word formulations will have low probabilities of occurring. Compare a manuscript that is being prepared for submission against the corpus and the machine should be able to point out the parts that may be either wrong or novel. Some kind of a Bayseian model would seem to be appropriate.

Now for natural language it is probably the case that there are not enough overlaps of complete sentences (though there may well be of phrases). However if you look at the academic literature then the scope of language used is very much reduced. The scientific literature in particular adopts an inbred subset of the English language, it's very own ghetto. One could image, for instance, taking all of the content of all articles published by Nature over the past 30 years, and use this as the control corpus. The person submitting a manuscript would get, on return of submission, a markup of where in their text there may be errors, with in addition perhaps, the most common forms of sentences that are found in their place.

I don't imagine that such a service would come into existence any time soon, but I think it would be cool. One could also use something like this to automatically recommend references or related papers. The "Journal Author Name Estimator":http://www.biosemantics.org/jane/ already does something like this for abstracts.

There is a wealth of research on probabilistic language models (see below), but I don't think anyone has tried out the idea proposed here.

It came to me after a few years working in a copy editing department of a scientific publisher. Again and again we would see the same kinds of corrections happening, and it just seems like an area ripe for automation.

"Using a probabilistic translation model for cross-language information retrieval":http://eprints.kfupm.edu.sa/74398/

"Language Analysis and Understanding":http://cslu.cse.ogi.edu/HLTsurvey/ch3node2.html

"A Parallel Training Algorithm for Hierarchical Pitman-Yor Process Language Models":http://www.cstr.ed.ac.uk/downloads/publications/2009/sh_interspeech09.pdf

A Bayesian network coding scheme for annotating biomedical information presented to genetic counseling clients "doi:10.1016/j.jbi.2004.10.001":http://dx.doi.org/10.1016/j.jbi.2004.10.001

"Phrase-Based Statistical Language Modeling from Bilingual Parallel Corpus":http://www.springerlink.com/content/b4ujx41571p47082/

"Bayesian Modeling of Dependency Trees Using Hierarchical Pitman-Yor Priors":http://videolectures.net/icml08_wallach_bmd/

"Using language models for tracking events of interest over time":http://boston.lti.cs.cmu.edu/callan/Workshops/lmir01/WorkshopProcs/Papers/mspitters.pdf

4 ways to set the heigth of your open social gadget

2010-02-08T00:00:00-08:00

You have a couple of options for setting the height of your gadget. I'll run through four methods that seem to work. First two words of caution, if you are using the eclipse plugin to develop on, you won't see the second method described here working. I don't know why, but the code works in iGoogle, so it must be something to do with the eclipse OSDE env. The second word of warning is that the syntax hilighter I'm using on this post chokes on CDATA tags, so the CDATA tags in the code examples are wrong. I've posted links to the sample gadgets at the end of this post, and you can get the samples there if you like. With those caveats, let us press on:

1. Don't do anything

sample 1 gadget code

The basic code below gives no specification for height. The container will default to 200px.

 


                title="t1"
            author_email="ian.mulvany@gmail.com">
        
    
    
    
    < ![CDATA[


Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

...

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.


    ]]>

All of the code after the content div in these examples is the same, so I'm omitting in the rest of the examples below.

2. Hardcode the height.

sample 2 gadget code

Pretty simple, tell the container exactly how big you content is going to be. For this example I knew that 400px was not going to be bog enough for the content that I wanted to display, so I added a scrollbar, easy!

 


       author_email="ian.mulvany@gmail.com" 
   height="400" 
   scrolling="true">
        
    
    

    < ![CDATA[

3. Use the OpenSocial dynamic-height library

sample 3 gadget code

Require the dynamic-height library, and then create a javascript function in your content section that calls the api. I've added in a handler that works on the window onload event, but this is probably not the best use of this method. You probably want to be doing it in response to some other event that will dynamically be changing the content of the gadget.

 


                title="t3"
            author_email="ian.mulvany@gmail.com">
        
        
    
    

    < ![CDATA[

4. Use the OpenSocial-jquery library

sample 4 gadget code

Almost exactly the same as method 3, except we use the opensocial-jquery library. If you are doing more with javascript this library seems to offer the simplicity of the jquery syntax, so for people who prefer this dialect of javascript it may be a very good option.

 


                title="t4"
            author_email="ian.mulvany@gmail.com">
        
   
    
    
    
    < ![CDATA[

What's the right amount to charge for phone content?

2010-02-04T00:00:00-08:00

Having a discussion earlier today with one of my colleagues, we were
discussing pricing models for iPhone applications. I buy a lot of apps
on my iPhone, but the content that we were discussing pricing for is
academic article content. This traditionally sells online for tens of
dollars per unit. If one wants to sell such content on an iPhone, but
at the same time retain the apparent value of the content how do you
price it?

It's a tough question, and unless you have a large stable of journals
of differential quality, it's going to be hard to run a series of
experiments to test change in sales volume based on price
differentials.

My gut feeling is that to drive large, or even modest volumes, of
sales on a device like the iPhone the content has to be priced below
the level at which the buyer is even considering the economics of the
purchase. It has to pass my "is it less than a cup of coffee" test. If
it is then I'm not even going to think about the opportunity cost for
purchase. Of course this is where it gets problematic for academic
content. You could only sell minor academic content cheaply (like news
coverage), or content with a reduced fucntionality, or you could try
and go for some sweet spot pricing (the cost of a large latte with
extra cream and caramel topping kind of thing).

The iPad is likely to allow academic publishers to go ahead and just
try to sell their content at the prices that they think they can
charge for on the web, but I'd still love to see a radical re-think of
pricing for at least some types of content on handheld platforms.

tags: iphone, ipad, publishing, science, pricing, coffee

Identity systems, science and the internet

2010-02-04T00:00:00-08:00

I just posted the following on a forum
(http://network.nature.com/groups/orcid/forum/topics/6547) on Nature
Network, I thought it was interesting enough to repost:

Bruce Schneier yesterday posted a very "interesting
article":http://www.schneier.com/blog/archives/2010/02/anonymity_and_t_3.html
that gets to the heart of identity systems on the web. The bottom line
of the article is that anonymity will always be a given as the price
for introducing verification or coining systems will always be too
high, and even if present they won't be effective. Of course it is
just exactly such a system that ORCID is proposing to introduce. I
think there is a chance that it will work, as I believe that the
incentive system in science is stacked in favour or real identities,
however it does point to the idea that rather than real identities,
when we deal with online transactions (be they knowledge transactions
or social transactions), it is rather the scientific persona that we
should be supporting rather than nessecarrily the real identity of the
person.

That idea ties in with a discussion that has been going on in network
about whether we should allow personyms, and I read this other "very
good piece":http://jonathanstray.com/identity-anonymity-and-controlling-trolls
on that topic in the past week too which supports the idea of allowing
personyms, so long as they can be ascribed a trust score, but baulks
at the idea of forcing users to use their real identity.

For the good foreseeable future scientific identities are going to be
tied to real identities, but does anyone know of cases where
scientists have done good, acknowledged work under a pseudonym? Could
we imagine such a thing happening in the future? I know that sounds a
bit far fetched, but it's happened already in the open source
community a number of times, notably with "why the lucky
stiff":http://en.wikipedia.org/wiki/Why_the_lucky_stiff.

tags: identity, science, ORCID

Cost benefit of publishing academic books,

2010-02-04T00:00:00-08:00

Nature has just published an editorial (http://www.nature.com/nature/journal/v463/n7281/full/463588a.html) promoting the idea of writing text books (http://dx.doi.org/10.1038/463588a).
Having worked for a number of years as a commissioning editor for a
major academic publisher, responsible for more that 20% of academic
book output, I have to say that the cost-benifit analysis for
publishing books ends up saying that it just does not justify the
effort for academics to write monographs. You probably won't see more
than a few hundred sales, citations to the work will be slow in
coming. Early career academics in particular are wasting valuable time
that should be spent on getting publications out. That said there are
three situations in which it might be OK to be involved in producing a
book:

1. You are at an advanced stage in your career and you want to codify
your vision of a particular subject. In such a case the work is a
labour of love, you have your laurels and now you want to produce an
artefact that synthesises your view on a topic. This is a highly
valuable exercise, look at the works of Chandrasekhar for an extreme
example of this. Of course, such an individual is going to go ahead
and do this anyway.

2. You have been instructing a class and have put together a detailed
set of instructional notes, especially for advanced classes in
graduate school. For a little more effort you can convert a large
batch of work that you have already done into another artefact that
can increase your academic reputation, go for it!

3. You are involved in a large consortium or working group. The act of
putting together a chapter for a book can cement working
relationships. What tends to be more important here though is the
collaborative process, more-so than the final artefact. The question
to be asked should be whether working with the given group of
academics is worth the time, rather than whether the final book will
be worth the time involved.

Banks and Credit Card companies are fuckers

2010-02-04T00:00:00-08:00

Via Bruce Schneir's blog there is a link to a paper about a new
security system that credit card companies are rolling out; a single
sign on system. The paper is here:
http://www.cl.cam.ac.uk/~rja14/Papers/fc10vbvsecurecode.pdf. The
bottom line is that the system is pretty insecure, but it is
incentivised by making the customer sign up to additional terms and
conditions that puts more liability on them than on the banks or
credit card companies. It seems a bit like adding a shrink wrap
licence around credit card usage. I think I would not mind if they
were increasing my security, but as outlined in the paper linked to
here, they are not. I recently had to use this system, and it seemed
very phishy to me, so I avoided it as much as I could, but for the
time being it looks like there is not going to be a way to avoid it in
future. Fuckers.

tags: online security, banks, single sign on.

what makes a chocolate "cheeky"

2010-02-02T00:00:00-08:00

I'm eating a chocolate bar which contains, I'm told, a 'cheeky caramel
layer'. It's shit like this that makes me despair of our modern
civilisation. According to the dictionary 'cheeky' means 'impudent;
insolent', and it has the synonyms 'saucy, audacious, bold'. I guess
that I am to feel that I have made a bold choice in opting for the
opulence provided by that extra caramel, but I can't raise the
enthusiasm to believe so.

opensocial development on eclipse, starting the web server

2010-01-29T00:00:00-08:00

Unable to retrieve spec for http://localhost:8080/test/gadget.xml. HTTP error 404

quick thoughts on the iPad

2010-01-28T00:00:00-08:00

I just commented over on the digatilist:

I'm in two minds, but on balance I'm optimistic. What they are
delivering is not a product, but a new platform, and one who's
limitation has the potential to drive innovation in the open web. What
I really don't like about what has been delivered is how closed the
system is, and how much this is not a tool that a developer could use.
I am in no way the target market for this product, but I don't care
that I'm not a market, I care about what I care about. The kinds of
things that I would want to do with such a device I can't do, however
applications on the web, in particular cloud storage + html 5
applications with local data storage, are fast moving to the point
where I could start coding in the cloud. My language of choice is
Python, there is already a toy implementation of a python interpreter
through a web interface (http://try-python.mired.org/). The Mozilla
foundation have a really advanced cloud based code editor
(https://bespin.mozilla.com/). Coding is about the biggest thing that
this device does not allow the power user to do. If those things are
close to being do-able in the cloud, then anything is possible.

What I do like about the device is the potential speed. We don't need
computers with fast processors. We need computers that feel fast. This
platform could really drive computer-human interactions towards the
cloud based model. It could really be a boost for rich web
applications (no need to go through the app store), and it could
really drive innovation in software and content delivery on the web.
It's almost a blank canvas upon which we may write or create the
interfaces that will define how people will interact with the content
or services we wish to deliver, and that's going to be a big deal.

tags: cloud, iPad, speed, web applications, html5

Usage of graph layouts on Biostor

2010-01-20T00:00:00-08:00

(co-author graph example)
inspired by

Climbing Goals 2010, review 2009.

2010-01-04T00:00:00-08:00

Looking back at my goals for 2009, I've not done very well, however my current level of fitness is pretty good, so I think 2010 is going to be a good year.

The main factors in holding back my climbing in 2009 were taking a few months off to get married and not being able to rent a car. Getting married was absolutely worth it, and the level of organisation and stability I have in my life now due to being married will only help in the long run. Although I got a driving licence last year, I was not able to rent a car in 2009 as I hadn't my licence for long enough, that will change now in 2010, and I should be able to make it out and about around the UK to get some cragging in.

So my goals for end of 2010 will be:

- fr 7a redpoint indoors
- V5 (I've done a few V4's towards the tail end of last year)
- a font 6a in font, preferably la marie rose (trip to font planned for April)
- Trad E1 (I've lowered this goal as I've had so little trad experience in the past number of years)
- A few trips to do some trad climbing.

I just got 9 out of 10 climbers make the same mistakes, and I hope that will give me some good training motivation for early in the year.

dreaming of snow

2009-12-01T00:00:00-08:00

DSC00458
Originally uploaded by Ian Mulvany

I just got some great feedback on this picture from someone on flickr, it was taken back in February when I was on a skiing holiday in Switzerland. The house was built by my wife's grandfather, and we came to it, it buried in a drift, the air as sharp as the icicles. It was a magical week, and I'm keen to return. At this time of the year thoughts turn to such things.

Keeping Dopplr and iCal in two way sync.

2009-12-01T00:00:00-08:00

I don't know if the following is going to work, but I'm going to try
it. Dopplr allows you to subscribe to your trips in iCal from a link
from the trips page. You can also publish a calendar locally to the
Dopplr servers that Dopplr will pull trips from. I've set up two
calendars in my iCal, one for pushing into Dopplr, and another for
pulling all Dopplr trips from. I hope this will let me add different
trips to Dopplr via different mechanisms, and retain the connection to
the push calendar from iCal without breaking it. That just gives me
more options for getting stuff into the one place.

The best option would be to have complete bi-directional sync, I'm
sure it will happen at some point, but in the meantime I'm not going
to sweat it.

tags: iCal, dopplr, organisation

Strange anxiety dream

2009-11-20T00:00:00-08:00

Last night I had a strange anxiety dream. I'm not usually prone to
blogging this kind of thing, but it was pretty odd. I dreamt that I
had been given command line access to a particle accelerator that was
embedded in that antarctic ice sheet. I'd run some kind of a job and
forgotten about it. Two weeks later I got a bill for £216, 000. I'm
not sure if that was the bill for the data that was generated, or for
the cost of the electricity for running the accelerator.

tags: dream, particle accelerator, antarctic

Notes from the Google Wave London roadshow

2009-10-31T00:00:00-07:00

Gtug blog post

On Monday I was at the London Google wave roadshow presented by Lars and Stephanie. The previous Friday I had helped to run a hack day for science applications in wave, so with that experience fresh in my mind I was particularly interested to see what they would say about a few particular topics.

We had a few problems in the hack day which could be summarised as follows:

- hard to debug with many points of failure when developing robots.
- the wave client is a hard interface for keeping track of conversations happening in the wave, and the document at hand.

If you could customize your client, and you could develop locally against your own server and robot endpoints, and the APIs were stable, then that would solve all the problems that we faced on Friday.

With that in mind I went into the meeting wanting to find out about the following

- status of open sourcing the client and server
- future api extensions
- usability improvements
- ability to deploy robots to non-appengine endpoints

All of these, and more, were discussed. It is an evolving platform, and Steph and Lars were very clear on their desire for community feedback to help target the dev resource that they have.

- status of open sourcing the client and server

Lars gave some detail on this. They intend to open source the server totally, they have already open sources the hardest part of the software in the server. They don't want to start opening up the client until the server specification is totally pinned down, as the server client protocol is currently changing a lot. There are also issues around opening up the client as parts of the client as it currently exist depend on proprietary Google technology, particularly search in the client. Lars said that they expect they will begin by progressively opening up parts of the client, for example the editor first.

I would like to see a library of units that could be dropped into a page to create a client. I would love to be able to redirect parts of a wave to different parts of the screen. One could then have a "Document" part of a wave and next to it a conversation part of the wave. One could also make something akin to a site specific browser for wave that could display a zoomed out view of the wave where you could see where every participant was at any one point. I guess that these kinds of interfaces would be possible.

- future api extensions

There were lots of tantalising bits and pieces mentioned throughout the talk. They were clear thy they want to extend the number if hooks that can be programmed against a wave. In no particular
order the following were mentioned.

A notification api so that changes in waves can be propagated out into other systems.

Very exciting was discussion of a regex hook. You could register your robot with this hook an instead of surfer (I guess that's the appropriate term for a wave user) adding a robot, on a regex triggering, the robot could be auto-added. One can think of many pitfalls with this, as well as many advantages, and they did say that they are taking their time and being very careful about how they implement this.

They are not working in integrating with email. This is the biggest requested feature, however they want to concentrate on making the wave experience great. They did look at this somewhat, but said that it changes the experience of using wave significantly. They expect other people to use the provided API's to figure it out but they are definitely not working on it themselves.

At the moment with a gadget that displays a view to something like a map, if one person moves the map all of the people can see that. They are thinknig about introducing a "wave view" and an "edit view". They are also thinking about tying the ability to change views in gadgets to the edit state of the surrounding blip. This makes a lot of sense, and I think is one of those things that only became apparent after a lot of people started using the system.

- usability improvements

Lots of discussion about various improvements. Things are rolled out to the sandbox first and then onto the main wave server. They will implement a draft mode, and introduce settings that can be used to set things like the level of spelling correction. Rosy is going to be coming, but probably only for short blips, in order to aid with conversation. I can't remember the other things that were mentioned right now.

Soon you will be able to remove people from waves, but I guess they want to make it a polite kind of a thing. Could lead to tit for tat behaviour.

They are working on wave gardening, where you can merge, split, concatenate, and generally de-threat the blips in a wave. This is going to be awesome of they can get both the protocol and UX right.

- ability to deploy robots to non-appengine endpoints

Definitely coming, probably before end of 2010. They are taking care with this, they want to be able to address issues such as what to do when the robot changes without a surfer knowing. This could be a security issue, and so they want to get this right.

Some other titbits:

They have on the order of 100s k users

Public waves are only .5% of all the waves created so far. (I guess lots of people kick open a few waves before even realising that they can make them public)

Largest wave is 100kb, this is a limit imposed by the system.

They have a list of top gadgets. They track usage, and are thinking about making a gadget gallery to increase discoverability of extensions and robots.

They are going to open up the embed api so that at some point in the future non-wave account holders can interact with waves through that api (on a web page for instance). They are not releasing this right now as they don't want to have the system slashdotted. From what I gather that all works, they are just waiting to scale.

The last two notes are that their developer resource is scaled according to how many people use the app. So if you want to see it developed more quickly, use it more! (If this is a general Google policy then they must run a pretty tight ship, as right now wave has 100s of thousands of users, and it's clear the dev team are having to make tough choices over what to build).

Shift + enter is your friend. Play with it when you are trying out editing and replying to waves!

Django models vs. Pickling objects for object persistence.

2009-10-19T00:00:00-07:00

I do a bit of code hacking and I often find myself putting objects into pickle files, and reading and writing them in order to fake object persistence. It's easy, but messy and it begins to leave lot's of pickle files sitting around in your path. I've decided to finally make the switch towards using a more grown up persistent object solution. I've decided to try out Django + SQLite. The former because I code in python, and Simon Willison uses it. The latter because I want to keep my system MySQL install pristine for another application that I am playing around with. I'm hoping that Django will hide the DB interface well away from me.

One of the key benefits that I hope to get out of this approach is to force myself into rigorously using Class based representations of model data, rather than naive python objects. I hope this will lead to cleaner and better documented code on my part. I also hope that a quick by-product will be to allow me to quickly prototype ideas onto the web via Google app engine.

We shall see.

An interesting invitation

2009-10-03T00:00:00-07:00

Last week an interesting invitation popped up in my email box, I got a
message from Richard Sterling asking if I would be interested in
previewing the program that the government are rolling out as part of
their open data project. They have set up a google group for
developers, and I guess I'm on the waiting list, or something. I'm
going to be interested to see what data sources surrounding science,
funding, teaching and publishing will be available there. I'm pretty
busy at the moment thinking about ways to improve Nature's online
offerings and make them more functionally useful, this might represent
an important path for that work.

The message says:

"It's really important to us that you are all comfortable with what we
are doing and how we are doing it, so let us know if there is anything
we can do to improve things."

I like that, sounds like a good approach.

Comparing Windmill to Safariwatir

2009-08-25T00:00:00-07:00

the post seems to be missing :/

Cron, python and Google Wave

2009-08-17T00:00:00-07:00

Documentation on getting a google wave robot written in python working with the Cron functionality is still a little sparse.

Getting a cron job into the capabilities.xml file.

Add the following directive to the main function of your python program:

myRobot.RegisterCronJob('/path',60)

When you look at you capabilities.xml file you should now see a new node in the file:

Why I think the Drake equation is a crock of shit.

2009-08-13T00:00:00-07:00

I had a brief discussion with a good friend and colleague this evening about the Drake equation. He has recently come to think that the equation can provide a good handle on understanding how many communicating civilizations exist in the universe, and how that reflects on the fragility of life on earth. A long time ago I had dismissed the Drake equation, feeling that it had very little to tell us about these matters. In face of his enthusiasm I thought it prudent to look again. I still feel that there is little of analytic value in this tool.

Before getting on, I will say that there are two very good aspects to the conversation about the equation. Firstly the question being tackled is a difficult one, and the idea of breaking it down into smaller more manageable components is an appropriate way to tackle such a complex issue. Secondly life on earth, all life, is fragile and precious, and taken far too much for granted by us, the only apparently self-reflecting species on the plant, as we sleepwalk our way from one human caused disaster to another. Anything at all that can bring the attention to the loneliness of our position in the cosmos, and perhaps through that engender a sense of solidarity in the face of a silent universe, is a good thing. Even if that thing is chimera, a vision beyond substance, an ideal.

That out of the way, I continue to feel that there is little informational value in the Drake equation. My sense is that our current limited understanding of many of the core components that make up the equation render it no better than guesswork. Moreover, the components could equally be divided in other groups without affecting how well the principle functions. I may well be out of date in my knowledge of what science can say about these questions. That would be wonderful. Let's go through the equation term by term. I'll try to argue that for each an order of magnitude difference in either direction could hold in the answer, and as such the range of possible answers provided by the equation as it is constituted is too broad to support much.

The equation is


N = R^{*} \times f_{p} \times n_{e} \times f_{l} \times f_{i} \times f_{c} \times L   \\
R^{*} -- the rate of galactic star formation
f_{p} -- fraction of stars with planets
n_{e} -- average number of life supporting planets
f_{l} -- fraction of these that go on to support life
f_{i} -- the fraction of these that go on to create intelligent life
f_{c} -- fraction of those civilizations that become communicative
L -- the length of time they hang around sending signals

OK, let's go through these one by one:


R^{*} -- the rate of galactic star formation
-- usually interested in sun like stars, as big start
-- dwarfs, which have a long incubation period, can capture/create planets

-- g type stars

This is given importance because the assumption here is that life has to be supported by planets. We could argue that life could arise in nebula where stars are almost formed, which are rich in heavy elements. I admit that this would still be functionally dependent on the star formation rate, but it would change the dependence of the final figure on the star formation rate by a considerable amount. It is quite far fetched to think of life like this, and it is clear that the bias in the equation is to find DNA based life like ours, liquid water, probably an oxygen atmosphere. I'm not sure that this is fair. We know from extremophiles on earth that it is possible for sulphur based life to exist on earth. It may be that what is needed is not a specific set of chemical elements, but rather the proper available energy transfer paths in a system so that a sufficient degree of local order can emerge. Of all the elements in the equation this is the one that I think is strongest candidate, and yet I'm not 100% convinced that it's effects can be simply determined.


f_{p} -- fraction of stars with planets

increase - binary systems can support more planets than we assumed
there are more - tight binaries



n_{e} -- average number of life supporting planets
f_{l} -- fraction of these that go on to support life

I'm going to roll these three into one discussion here. As far as I can tell, we don't know how life originated on earth. We do know the relative abundance of the building blocks of carbon based life in the universe. From studying line emissions we can see that relatively complex molecules, such as alcohol, and the bases that form DNA are pretty common, so the soup that formed life could be expected to be in quite a lot of places. I remember recalling that self-replicating forms of molecules were often low-energy states, boosting the likelihood of the emergence of such forms in a random process than one might naively think, but without a model for how to make the kind of life that we know we are yet to be able to put a lower bound on this figure. Then there are all of the potential non-carbon based life's that might emerge, driving this number upwards.


f_{i} -- the fraction of these that go on to create intelligent life
f_{c} -- fraction of those civilizations that become communicative

Another approach might be to look at the cross section of what
I haveMy feeling is that the underlying assumption of what could life subcomponents

In discussing this briefly with Adam this afternoon I thought it would be good to go over my thoughts on this again.

OK, it's not that bad.

Swine flu t-shirts

2009-07-29T00:00:00-07:00

I'm sitting at home in London with a head cold. For the dew minutes it
took me to check my temperature (37 thank you very much) I was
wondering whether it might be the old swine flu, but no, just a normal
head cold.

Made me think, someone should make a t-shirt with a biometric sensor
that checks on true wearers' temperature. No fever and it would say
"don't worry it's only a cold". Fever and, well there are lots of
things it could say.

Fluid Dynamics in the Browser

2009-06-11T00:00:00-07:00

The Navier-Stokes equations:


\frac{\partial \mathbf{v}}{\partial t} + ( \mathbf{v}\cdot\nabla ) \mathbf{v} = -\nabla p + \nu\Delta \mathbf{v} +\mathbf{f}(\boldsymbol{x},t)


\frac{\partial \mathbf{v}}{\partial t} + ( \mathbf{v}\cdot\nabla ) \mathbf{v} = -\nabla p + \nu\Delta \mathbf{v} +\mathbf{f}(\boldsymbol{x},t)

Describe the flow of an incompressible fluid. The equations are non-linear and are notoriously difficult to solve. I've always found them to be very beautiful and the fact that Sr George Stokes was an Irishman probably had some effect on my taking to the subject when I was an undergrad.

There are many subtleties hidden in the equations, notably the issue of conservation of momentum and the appearance of turbulence. The subject inspired the following rather fun ditty:

Big whorls have little whorls
That feed on their velocity,
And little whorls have lesser whorls
And so on to viscosity.

-- Lewis F. Richardson

When I was an undergrad I spent some time playing with a Cray supercomputer running simulations of fluid flow on a sphere. A supercomputer. A big one. In my grazing of the online new sources today I saw a link to Oliver's fluid dynamics simulation which provides a Navier - Stokes solver in the browser in javascript. It's pretty, and it's damn impressive.

It doesn't solve the actual equations, but a diffusive model optimized for stability, speed and aesthetics based on this paper.

It really shows the potential of the browser as a tool for the teaching of science. Being able to make changes to your code and test the results right there, nice!

unknown post

2009-05-20T00:00:00-07:00

In February 2009 a mac security update broke my perl installation. You can read more about that on this Slashdot article (http://hivelogic.com/articles/view/top-10-programming-fonts) , ulimatly forcing me to reinstall my entire system. One of the main reasons for this is that I had a hevily customized perl setup as part of installing Connotea and I based this on top of the system perl. This was a bad idea, but was the quickest way to get Connotea up and running on my local machine at the time.

Having learnt my lesson I decided to set up Connotea on a totally sepearte perl install. The following instructions assume a totally clean machine to begin with. You will need to have the Apple Developer Tools installed.

For installing on a clean nix please follow the instructions in the main README file.

Connotea is written in perl and uses mod_perl under apache to talk to a MySQL datastore. The code uses memcached to help with preformance. It uses a large number of perl modules that need to be installed before the software can run. This guide will run through setting up all of the required depedancies.

Before getting to installing Connotea let’s set up our dependancies.

A word on MacPorts vs compiling your own.

Bauhaus has a very nice description on how to set up a dev environment in OS X, however the description here depends hevily on MacPorts. MacPorts is an easy way to install software onto your mac, and it will do so in a way that will not conflict with system installs, however it provides the wrong architecture for perl for Connotea, so you will have to install perl and all of the perl dependancies seperatly. This is a bit of a shame, as installing the dependancies is much faster under MacPorts than it is under CPAN. We can install some of the required architecture under MacPorts and that is what we will do. One last note about MacPorts. It will install a very large amount of dependancies onto your system. This is expected, though scary, behaviour. Don’t worry.

In order to use MacPorts you can install an applicationportsapp. After installing you can get the port version of a given piece of software with hte following command:


$ sudo port install port-name

where port-name is the software you want to install. MacPorts creates it’s own path system and will install into

$ /opt

MySQL

Download the package from the MySQL site and follow the instructions for installation. For a quick cheat sheet of common commands for MySql under OS X there is a great MySQL Reference for OS X over at Comentum.

After installing the most important commands are to set an admin password, to start the server and to stop the server.

setting the root password: $ /usr/local/mysql/bin/mysqladmin -u root password sniggle

starting and stopping from the command line: $ sudo /usr/local/mysql/bin/mysqld_safe –user=mysql &

If you wish to set preferennces for MySQL when it is running you can create a conf file. This should be under /etc and should be called my.cnf.

Make sure you know where your socks are.

There is a known issue with MySQL under OS X in that the location for the socket file is not where OS X thinks it is. There are a number of solutions to this.
Edit the my.cnf file so that the mysql server creates a sock file in a location of your choosing. Reccomended like so:

mysqld socket=/tmp/mysql.sock

Now if you are using a perl installed via MacPorts and you want to use a non-mac ports version of mysql then you might encounter the following problem with the DBI module.

failed: Can’t connect to local MySQL server through socket ‘/opt/local/var/run/mysql5/mysqld.sock’

What is going on here is that the DBI module is reading a setting telling it that the mysql socket should be found in ‘/opt/local/var/run/mysql5/mysqld.sock’. I’m not sure where it gets this setting from but there are a couple of options.

Use a mac ports version of mysql. See

http://www.hennessynet.com/blog/?p=40

http://www.martinoflynn.com/blog/2008/08/13/installing-mysql-on-mac-os-x-105/

http://2tbsp.com/content/install_and_configure_mysql_5_macports

Specifically tell the DBI module where to look for the socect. In any perl scripts that call the DBI module you can use the following command to specify the location of the socket file:

$ENV{MYSQL_UNIX_PORT}=’/tmp/mysql.sock’;

Place this in your script after you call the DBI module.

You can alias your actual sock file to the location that the DBI module is expecting it, via:

$ sudo ln -s /tmp/mysql.sock /opt/local/var/run/mysql5/mysqld.sock

you may have to create the /opt/local/var/run/mysql/ directory.

You may wish to create an alias so that this link does not get removed on server start up.

Apache2

This is where MacPorts comes in handy. To install apache2 all you need to do is

$sudo port install apache2

This will install a number of dependancies on your system, and it will install apace2 into /aop/local/apace2. You can start apache2 and ensure that it starts at next startup with the follwoing command:

$sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist

You can test that it works by pointing your browser at http://localhost/ and you should get an “It Works!” message!

If you want to start and stop apache2 while working on the configuration of the server, or to check your sanity then you can use the following cammands:

$ sudo /opt/local/apache2/bin/apachectl stop $ sudo /opt/local/apache2/bin/apachectl start

memcached

Using the Macports version this is pretty easy.

sudo port install memcached

A few words about CPAN

There is a comprehensive description of life under CPAN on OSX however I decided to wing it. I dislike CPAN. It is a very attention demanding program that has a tendancy to want a lot of attention. It is nice to know that you can install modules from the command line with the following syntax:

$sudo cpan -i -f URI::QueryParam

This means if you can get a list of dependancies together then you can semi-automate the process of install them all. I understand that the preferred method of managing a large selection of perl dependancies to is create a CPAN Bundle, but I didn’t manage to do this, mainly due to a lack of familliarity with CPANishness. It is also worth noting that if CPAN is not able to install a module you can always have a go at downloading and installing the modeule from source. This is often quite straightforward.

perl

As mentioned earlier, MacPorts do not porvide the correct type of perl for running Connotea. Connotea requires ‘PerlOptions +Parent’ in the Apache configuration file. This requires a multithreaded perl. MacPorts only provides a perl without threads. This means you have to build your own perl. You can check which version of perl you have installed with the following command

$perl -V

What you want to end up with is the following:

Platform: osname=darwin, osvers=9.6.0, archname=darwin-thread-multi-2level

Go ahead and grab a copy of Perl. You pretty much want to follow the instructions given in the README.macosx file. You want to make sure that you build perl with multithreaded support. I followed a lot of instructions for passing arguments to the Configure script without much success. In the end I think that my mistake was passing the ‘-de’ flag which tells the script to accept the default settings. Don’t do this, just go through the interrogation and at some point you will be asked if you want to build with multithreaded support. Say YES!!. Hopefully you will end up with a perl now installed in /usr/local/bin/perl. This will be independant from the system perl and you will no longer be sucseptible to Apple fucking you over (allbeit unintentionally). Now when you build anything on top of this perl it will have the same architectural support (I believe), so you can now make mod_perl on top of this perl. One tip for speeding up the process, if you have a multi-core machine you can pass an argument to make which will tell it to use the number of cores that you pass it to, so

$make -j 2

will make twice as fast as make.

mod_perl

Get a copy of mod_perl. Go into your mod_perl source directory and from there do the following:

$perl Makefile.PL MP_APXS=/opt/local/apache2/bin/apxs $make && make test

after a lengthy build process you should be ready to install with:

$sudo make install

In /opt/local/apache2/modules you should now find mod_perl.so. We can now test our mod_perl to see if everyting is OK. Have a look at the fast guide to getting started with mod_perl and follow the instructions there.

To set up a test of mod_perl we need to edit apache’s httpd.conf file which can be found: /opt/local/apache2/conf/httpd.conf

Add the mod_perl module by adding the following line to the conf file

LoadModule perl_module modules/mod_perl.so

Stop and restart your apache2 server and take a peek at the error log, you should get a line like the following:

$tail /opt/local/apache2/logs/ Thu Apr 23 14:59:22 2009 Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.8j DAV/2 mod_perl/2.0.4 Perl/v5.8.9 configured – resuming normal operations

yay!

perl dependancies

OK, Connotea requies a lot of dependancies, most of which can be installed through CPAN, a few which need to be manually installed. I’ve provided a file 1.8.extended.deplist.txt that contains what I believe to be the current set of modules that Connotea requires. If you pass the list of names into check-modules.py then you can check whether these modules have been installed. Passing any argument into the script will produce verobse output. Passing no arguments will just list the modules that have not been installed.

For the time being please ignore the following modules:

Apache2 Apache::Const Apache::File File::Touch Net::OpenID::JanRain::Consumer

$more 1.8.extended.deplist.txt | ./check-modules.py verbose

You can generate a shell script to install non-installed modules via CPAN with the following command:

$more 1.8.extended.deplist.txt | ./check-modules.py | awk ‘{print “sudo cpan -i ” $1}’ > install-modules.sh

and then:

$chmod u+x install-modules.sh $sudo ./install-modules.sh

This will take some time. The CPAN interface is slow and requires a lot of intervention. If you get stuck with CPAN for a given module, then download the module and install from source.

I had to manually install Authen::Captcha http://search.cpan.org/dist/Authen-Captcha/Captcha.pl

Connotea code

The most recent public version of the Connotea code is now on GitHub under the name connotea-public. The main working branch is still under darcs, however we are in the process of switching over to git as the time to upload local patches to the dev server is becomming prohibitive. We may move to mercurial at some point in time, but for the time being the public facing snapshot will remain in git. Head on over and grab a copy. The main files of interest for continuing the setup will be in /connotea-public/sql/, /connotea-public/README, and /connotea-pulic/config

Setup MySQL databases and permissions

Connotea uses a four databases. The main content database, the search database which is a replica of the content database, a click tracking database and the wiki database. The content database is a MyISAM database, and the search database is an InnoDB database. The schema for the content database is provided by schema.sql in the sql directory. To generate the search database do the following:

$perl mkschema_search < schema.sql > schema_search.sql

Now set up the schema of the content database and the search database:

$mysql -u root -p < schema.sql $mysql -u root -p < schema_search.sql $mysql -u root -p < clicks.sql

You need to set up the search database to be a slave of the content database. This is done in the MySQL conf file by adding the following directives:

  [mysqld]
  # local replication of bibliotech to bibliotech_search:
  server-id=1
  log-bin=mysql-bin
  binlog-do-db=bibliotech
  replicate-same-server-id=1
  replicate-rewrite-db=bibliotech->bibliotech_search
  replicate-do-db=bibliotech_search
  master-host=localhost
  master-user=search_repl
  master-password=pass

You may want to add the folloing directive too:

  # change stopwords in support of bibliotech freematch feature:
  #ft_stopword_file=/etc/mysql_stopwords.txt
  ft_min_word_len=2
  ft_max_word_len=255
  # allow packing of queries
  group_concat_max_len=8192

You now need to set up the appropriate permissions form within MySQL for the following users ‘conwiki’, ‘search_repl’ and ‘connotea’:

mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, ALTER ON bibliotech_search.* TO -> connotea@localhost identified by ‘secret’;

mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, ALTER ON bibliotech.* TO -> connotea@localhost identified by ‘secret’;

mysql> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON . TO -> search_repl@’localhost.localdomain’ IDENTIFIED BY ‘secret’;

We now need to set up the wiki database. This is done through the following command:

mysql>CREATE DATABASE conwiki; mysql>GRANT ALL ON conwiki.* TO conwiki@localhost IDENTIFIED BY ‘secret’;

To set up the schema we just let wiki-toolkit-setupdb have at it:

$/usr/local/bin/wiki-toolkit-setupdb –type mysql \ –name conwiki \ –user conwiki \ –pass secret \ –host localhost

Remember to populate the “COMPONENT WIKI” block of your configuration file with the wiki database details.

Connotea will be informed of the database that it needs to connect to in the conf file. I would say that it goes without saying that the passwords all need to be congruent, but I keep messing up that part, so I’m not going to say that.

Setup Apache directives and paths to the code.

Ok, now we have mysql sitting there waiting to act as a data store. We have the connotea code. We just need to connect these together and to the outside world through the magic of apace and mod_perl.

You need to add a directive like the following to your apace conf file. Remeber, that file should be in /opt/local/apache2/conf/httpd.conf

change the document root, as we are going to host connotea from

a more traditional location

DocumentRoot “/opt/local/apache2/htdocs”

DocumentRoot “/var/www/html”

ensure that we can follow symlinks:

REXML could not parse this XML/HTML: 
<Directory "/var/www/html">
    #
    # Possible values for the Options directive are "None", "All",
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that "MultiViews" must be named *explicitly* --- "Options All"
    # doesn't give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.2/mod/core.html#options
    # for more information.
    #
    Options Indexes FollowSymLinks

    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be "All", "None", or any combination of the keywords:
    #   Options FileInfo AuthConfig Limit
    #
    AllowOverride None

    #
    # Controls who can get stuff from this server.
    #
    Order allow,deny
    Allow from all

</Directory>

add directives for connotea

REXML could not parse this XML/HTML: 
<VirtualHost 127.0.0.1>
        ServerName local.connotea.com
        ServerAlias *local.connotea.com
        ServerAdmin ian@mulvany.net
        DocumentRoot /var/www/html/connotea
        PerlOptions +Parent
        PerlSwitches -I/var/www/perl/connotea
        PerlModule Bibliotech::Apache
        PerlModule Bibliotech::AuthCookie
        <Location />
          SetHandler perl-script
          PerlHandler Bibliotech::Apache
          PerlAuthenHandler Bibliotech::AuthCookie::authen_handler
          AuthName Bibliotech
          AuthType basic
          require valid-user
          #ErrorDocument 503 /paused.html
          #ErrorDocument 503 /readonly.html
          ErrorDocument 503 /unavailable.html
        </Location>

REXML could not parse this XML/HTML: 
</VirtualHost>

In your /etc/hosts file add the following line:

1 local.connotea.com

This means you can point your browser at local.connotea.com and see the local installation.

in /var/www create perl, html and site

in /var/www/site create the directory connotea and place the source code in there.

create the following symlinks:

/var/www/perl/connotea -> ../site/connotea

and

/var/www/html/connotea -> ../perl/connotea/site/npg

Make sure that the paths to the code is readable by apache. That also means that the full paths have to be readable by apache.

Set up the connotea conf file.

copy /connotea-code/confi to /etc/bibliotech.conf

The most important declarations to set up are the following:

DOCROOT = ‘/var/www/perl/connotea_code/site/default’ LOCATION = ‘http://www.mydomain.com/’ DBI_CONNECT = ‘dbi:mysql:bibliotech’ DBI_USERNAME = ‘user’ DBI_PASSWORD = ‘secret’

just the database name of the replicated MyISAM FULLTEXT-enabled database

DBI_SEARCH = ‘bibliotech_search’

for debugging purposes set EXPLAIN_HTTP_CODES to true

set it to false when you have everything running.

It causes application errors to be printed to the requesting page.

EXPLAIN_HTTP_CODES = true

CLICKS { # options regarding click tracking # database connection details DBI_CONNECT = ‘dbi:mysql:clicks’ DBI_USERNAME = ‘user’ DBI_PASSWORD = ‘secret’ }

COMPONENT WIKI { #DBI_CONNECT = ‘dbi:mysql:conwiki’ #DBI_USERNAME = ‘conwiki’ #DBI_PASSWORD = ‘secret’ #ADMIN_USERS = ‘admin’ #LOCK_TIME = ‘10 MINUTE’ #ALLOW_EDIT = true #HOME_NODE = ‘System:Home’ # page size limit is in characters #MAX_PAGE_SIZE = 40000 # maximum external hyperlink count, cuts down on spam #MAX_EXT_LINKS = 75 # scan: 1 means check text against ANTISPAM > TAG_REALLY_BAD_PHRASE_LIST # scan: 2 means check text against ANTISPAM > WIKI_BAD_PHRASE_LIST+TAG_REALLY_BAD_PHRASE_LIST #SCAN = 1 # to admit the spam rule that rejects wiki text, set this to true: #SAY_SPAM_RULE = false }

sendmail needs to be set to the OS X path.

SENDMAIL = ‘/usr/sbin/sendmail’

Final adjustments

Ensure that antispam_score.csv is writable by connotea.

Turn it on.

You should be able to run the application now.

Gotcha’s

In a previous installaion I had problems getting the perl GD module installed

You need to downgrade LWP, use the following to find out which version of a perl module you have installed. Follow the instructions at this site to get this installed:

Another great example of how being lazy can reap great rewards

2009-04-16T00:00:00-07:00

Last year I got really obsessed with the idea of making my google email contacts list, my phone list and my desktop address book all sync together. I couldn't find a simple to use free application that would do all of this for me. For a while I was using Spanning Sync to keep calendars in sync. I had a Nokia N95, my desktop machine is a mac and my online life is mediated by Google. I thought about expending energy to write my own solution, and would think about it from time to time, but today I am happy to announce that I needn't have bothered. Now with my iPhone and <a href="http://www.google.com/mobile/apple/sync.html">Google mobile sync</a> everything seems to just work, at least for the time being. I'm pretty happy about this right now.

tags: iphone, google, contacts, sync, lazy

notes on "Classes of complex networks deﬁned by role-to-role connectivity proﬁles"

2009-04-14T00:00:00-07:00

Looking for algorithms to do community detection (there are a bunch from the famous Girvan-Newman algorithm doi:10.1073/pnas.122653799 which seems to have gained popularity through being very simple, effiencet and usually pretty powerfull, through to a method doi:10.1103/PhysRevE.71.046117 by Ziv, Middendorf and Wiggins. I have to admit I don't understand that paper yet, but hope to come back to it.) I was pointed to a paper by Guimera, Sales-Pardo and Amaral. In looking for the paper the first one that I found was Classes of complex networks deﬁned by role-to-role connectivity proﬁles on DOI:10.1038/nphys489. It turns out that they describe their algorithm in the doi:10.1038/nature03288, but as I encountered the nautre physics aritcle I may as well read my way through that and give you the skinny on it.

The main story in this paper is that real networks are complicated, and that the properties of modules in the network are not reflected by the average properites of the entire network. The implication would seem to be that models for generating netowrks do note capture importnat properties of real world netowrks. That's not really a great surprise, however I guess that it is nice that someone has sone an explicit comparison.

One interesting idea in the paper is that they try to classify modules according to how the average connectivity of the connecting nodes in the modules (the nodes that bridge out to other modules) is distributed among the other modules. The classes they describe are non-hubs, with non-hubs being further described into ultra-peripheral nodes, peripheral nodes, satellite connectors and kinless nodes. Hubs get broken out into connector hubs, global hubs and provincial hubs.

These descriptions are tightly defined in terms of properties of the identified modules. The claim in the paper is that global properties of real networks, specifically some degree-degreee correlations, can only be reconstructed by taking into account this modular structure. The new property is called the "Participation Function".

There is another imporant aspect to this picture. If we think of the modules as building blocks, this paper is saying that the kinds of building block of the network are key to understanding the properties of the network. In addition, how those building blocks tend to connect to each other is an equally important aspect of this picture and can be measured by looking at the likelihood of a connection between two blocks in a given network in contrast to a random connection probablility.

An illuminating example is given by describing the differences between Cincinnati airport and Johannesburg airport. The two have the same degree distribution, but you can fly to the latter from most major airports in the world and not the former. This is described by noting that Joberg is the most connected city in it's region, but the same does not hold true for Cincinnati (trivia, this city was named after a roman dictator).

This is mainly a descriptive paper, but the authors suggest that the differences may arise in real world networks depending on whether they are transportation networks that are constrained by conservation laws, as opposed to signialling networks that face no such constrainsts. (I'm not sure, without thinking about it further, that you wouldn't be able to define some kind of a conservation law on a signalling network, given that there will tend to be some constrainst, e.g. attention time/information processing capacity of the reciever nodes).

It seems from the paper that they use simulated annealing to determing modularity.

The math bit

The modularity of a partition $\mathcal{M} \left( \mathcal{P} \right)$ is defined as


\mathcal{M} \left( \mathcal{P} \right)  = \sum_{s=1}^{N_{M}} \left[ \frac{l_{s}}{L} - \left(  \frac{d_{s}}{2L} \right)^{2} \right]

Where $L$ is the number of links in the network, $N_{M}$ is the number of non-empty modules in the partition, $l_{s}$ is the number of links in a module $s$ and $d_{s}$ is the sum of degrees in a module $s$.

Pick your favorite algorithm (I don't have one yet!) to find a partition to your liking.

Modules roles are determined by looking at where the modules fall in the $z$ - $P$ plane where $z$ is the relative within module degree and $P$ is the participation coefficient. I'll have a look at these two metrics in a later post.

Why measure scientists and their work?

2009-04-14T00:00:00-07:00

I'm interested in using graph theory to develop tools to improve science (whether I manage to do this is a moot point, I believe that someone will, and that such tools will be useful). One of the questions raised by any effort to introduce new systems that may be used to compare and contrast the work of scientists is whether such systems can have any net benefit when weighed against the potential pressure that they may place on working scientists. A large amount of discussion has taken place in the Citations in Science forum. Other places where there is a discussion about the kinds of tools that are being considered is the International Repositories Infrastructure wiki, and again over at Nature Network Martin Fenner is asking a few questions about author identifiers.

Themes that frequently bubble up are:

awarding credit, particularly for non-journal writing contributions

disambiguating the literature, especially for non-ascii authors

tracking trends in the literature

matching grant funding to research output

opening the monopoly of authority held by large journals and citation indexing services

There are other themes too, and I may amend this post as those themes come to me.

posterity vs a living wage

inbound vs outbound effects on the scientist.

Neat blog on Sankey diagrams.

2009-04-14T00:00:00-07:00

I just noticed a very nice blog dedicated to Sankey diagrams. As Phineas mentions in one post: Sankey diagrams are directed graphs.

Why I'm setting up this blog

2009-04-13T00:00:00-07:00

Right, well, I'm pretty interested in graph theory. I work in publishing for the journal Nature on new web products, and I need a focussed way of getting my thoughts together on a range of topics that touch on science. I wanted a space that was more focussed than my personal blog.

I have a few specific aims for this blog.

Get some of my thoughts on publishing, science and the philosophy of science out of my head and into an archived public space

Make notes on any academic papers that I read that touch on my area of interests

Motivate me to take all the cruddy code that I have lying around and push it public

I'd like to use this blog as a driver to learning how to visualize and extract sense out of large graphs

I'm hoping to perhaps get to grips to Processing as a means for the last of these three.

In regards to the first of these three, I do intend to draw a dotted line between my personal opinions and bits of science that I find interesting, and projects that I am involved in at work. I like the tao of mac disclaimer, and I hope to tend more towards following this advice than not, however it is not clear to me that I would wish to refrain from commenting on aspects of the work that I do, especially those that reside very much in the public domain.

That said, all opinions voiced here are strictly my own.

Moving over to the googleverse

2009-04-13T00:00:00-07:00

Well, today I finally made a switch that I had been thinking about for a long time, I moved the hosting of my personal domain to google app engine with email being managed by google apps for my domain. http://www.mulvany.net is now a google app engine site. At the moment there is not much content there, but the google app engine tool allows me to run a perfect cop of my site off line, and to edit locally and then push live. That's really nice, and I liked this interaction model while I was creating my wedsite (wedding website). Prior to this my site was running on Zope. To be honest, editing it was a pain in the ass.

My google app engine setup is using the standard out of the box Django setup. To be honest, getting uri's connected to funtions or to content is not as transparently easy as with sinatra on ruby, but it's pretty nifty, so I'm a happy bunny.

I was worried about email, but the MX records took about 30 mins to switch over, so no sweat, it was pretty smooth.

There is one downside, i'm hosted on networksoloutions, and it's looking like those bastards don't allow proper 301 redirects, so I may not be able to get http://mulvany.net to point to http://www.mulvany.net. I've gotten a domain renewal until 2013, so I might just have to suck it up. There is a discussion over at http://blogging.nitecruzr.net/2007/11/vagaries-of-publishing-your-blog-to.html pointing out some of the problems about this.

I also set up a, well, I'm hesitant to call it work related, let's call it a blog about the stuff that I'm interested in that's not totally random, but that does have a relationship to my interests in science. It's over at Directed Graph, it's a bit new, and it probably won't be updated any more frequently than this blog, but I'm going to try to keep the content over there a bit more focussed on topics around science, graph theory and publishing.

Grave of the reverend Thomas Bayes

2009-04-13T00:00:00-07:00

I visited bunhill fields at the weekend. It's a non conformist
graveyard just off of moorgate in London. Over the course of it's
active life over 120,000 people were buried there, in a space a little
under four acres. Notable residents include Daniel Defoe and William
Blake, but the occupant I was most intrigued to see was Thomas Bayes,
the founder of Bayseian statistics. The large tomb contains
thebesmains of many members of the Bayes family. It reminded me of a
time when I passsed by the grave of Stokes on a graveyard in
Edinburgh, also old and greened and sentinalled by ancient trees. So
many great minds have fed the soil of this little island.

Directed Graph blog setup.

2009-04-13T00:00:00-07:00

I've decided to use syntaxhiligher2 to do syntax hi-lighting and a call to Yourequations.com for rendering latex on the blog. Bot Cyborg has a good description of how to get this to work on blogger.

The main advantage of this approach is that the pseudo code is written in the blog posts in plain encased in a pre statement, and the equations are also just vanilla latex in a pre statement. So if we wanted to think about the betweenness centrality of a graph we have the equation:

\[ C_B(v)= \sum_{s \neq v \neq t \in V \atop s \neq t}\frac{\sigma_{st}(v)}{\sigma_{st}} \]

and in python for a given graph object we could use the excellent networkx package to calculate it like so


import networkx as nx #import networkx
G=nx.read_adjlist("graph.adjlist")  # read in a graph graph
nodes_betweenness_centrality = nx.betweenness_centrality(G) # returns a dictionary
for node_betweenness_centrality in nodes_betweenness_centrality:
    print node_betweenness_centrality

testing latex and syntax hilighting on blogger

2009-04-12T00:00:00-07:00

Some python code

some links
http://www.ramymostafa.com/?p=99
http://alexgorbatchev.com/wiki/SyntaxHighlighter
http://www.botcyb.org/2008/10/rendering-latex-in-blogger.html
http://blog.yourequations.com/

Syntaxhiligher2 (unreliable)



def AuthenticateSession(username,password):
     blogger_service = service.GDataService(username,password)
     blogger_service.source = 'mulvanynet-BloggerTagger-1.0'
     blogger_service.service = 'blogger'
     blogger_service.server = 'www.blogger.com'
     blogger_service.ProgrammaticLogin()
     return blogger_service

def PrintUserBlogInfo(blogger_service):
     query = service.Query()
     query.feed = '/feeds/default/blogs'
     feed = blogger_service.Get(query.ToUri())
     blog_id = feed.entry[0].GetSelfLink().href.split("/")[-1]
     print feed.title.text
     print blog_id
     for entry in feed.entry:
         print "\t" + entry.title.text,
         print entry.GetSelfLink().href.split("/")[-1]

def GetBlogFeed(blogger_service):
    query = service.Query()
    query.feed = "/feeds/default/blogs"
   feed = blogger_service.Get(query.ToUri())
   return feed


\int_{0}^{1}\frac{x^{4}\left(1-x\right)^{4}}{1+x^{2}}dx
=\frac{22}{7}-\pi

renders as


\int_{0}^{1}\frac{x^{4}\left(1-x\right)^{4}}{1+x^{2}}dx
=\frac{22}{7}-\pi

Getting listed on nature blogs

2009-04-12T00:00:00-07:00

I'm gittin meself listed on nature blogs. I better whup this blog back into shape first though.

Adding a tag cloud to blogger

2009-04-12T00:00:00-07:00

I've been thinking about neat ways to make blogs approach research articles; about how to make some of the core features of a journal be more apparent in a blog, which has led me to think about blogger widgets. That led me to the Phydeaux3 blog where there is a kick ass implementation of a widget to display a tag cloud for blogger. Hopefully you can see that it works pretty well. Neat plugin.

Hopefully I'll be able to dissect this to put in place some of the ideas I have regarding journals.

I woke up today and git-svn was broken.

2009-03-10T00:00:00-07:00

Odd, it was working last time I used it. Today I got the following error

git svn
IO object version 1.22 does not match bootstrap parameter 1.23 at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/XSLoader.pm line 94.
Compilation failed in require at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/IO/Handle.pm line 263.
BEGIN failed--compilation aborted at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/IO/Handle.pm line 263.
Compilation failed in require at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/IO/Seekable.pm line 101.
BEGIN failed--compilation aborted at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/IO/Seekable.pm line 101.
Compilation failed in require at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/IO/File.pm line 133.
BEGIN failed--compilation aborted at /System/Library/Perl/5.8.8/darwin-thread-multi-2level/IO/File.pm line 133.
Compilation failed in require at /usr/local/libexec/git-core//git-svn line 40.
BEGIN failed--compilation aborted at /usr/local/libexec/git-core//git-svn line 40.

The following blog post http://blog.grogmaster.com/2008/12/notes-about-git-svn-google-code.html seems to reference some kind of git-svn problem on Mac, though whether it is related to the issue that I am having I don't know. I've decided to try to follow the advice there and install through Mac Ports, which involves installing (dmg download available from the site). After that I tried

sudo port install gs-utils

For a laugh. I'm now waiting for all of the mac ports dependancies to load and install. I figure I've got about a 15% chance of this going through without breaking, given the current state of my system. I'm off for a coffee now, I'll check back later and see what happens.

Rail in the UK is really pretty shitty

2009-03-09T00:00:00-07:00

Last month I bought some rail tickets online, and got to London Bridge only to be told that I had to go to Liverpool Street in order to print out the tickets. Five minutes before my train was due to depart. I had to buy a new set of tickets for that days journey, and get up extra early the next day to make it to Liverpool street to print out my tickets.

This was really annoying me, so I wrote a pretty rude email and got a response saying that if I returned my tickets with a cover letter I would get a full refund, anyway, I'm about to send off for a refund, the cover letter is reproduced below:

09/03/2009

Dear Madam,

I am returning the pair of unused tickets for a journey on the
14th of February 2009.

I purchased these return tickets online for a journey between
London Bridge and Lingfield. When I arrived at London Bridge I
was informed that I had to go to Liverpool Street Station to print
my tickets, and that there was no way to be issued with my tickets
at London Bridge. I was forced to purchase another set of tickets
for journey on that day.

The following day I had to stop by Liverpool street in order to
print the tickets for travel on the 15th, and in addition I printed
the unused tickets, which I am returning in this letter.

The idea that you would create a system which allows for purchase
and collection of tickets via a web interface, but at the same
time not provision one of your busiest metropolitan hubs with the means
for retrieval of those tickets is astonishing. I understand that investment
in the ticketing machines may be large, but to roll out a system that
is so patently broken cannot be in the best interest of the consumer.

At the very least, as an interim measure, providing tickets that may be printed online could be a viable solution. Every rail company that
I have travelled with across Europe has such a system. National Express bus
service has such a system. Even Ryan air uses such a system, which indicates to this mind that it cannot be too expensive to implement.

Sincerely Yours,

- Ian Mulvany

other tips for new york

2009-03-02T00:00:00-08:00

Also Mona's bar on 13th and b (a bit of a dive, but all the better for it)
The east village/alphabet street areas have great bars, there used to be one called the drugstore.

Best Bagel: H&H on 75thish and broadway

Ah, sure, it's not a bad wee city.

Oh, the museums, they are pretty good too.

The Metropolitan (it's free)
The Frick Collection
- Guggenheim (depending what's on)
MOMA
MOMI

Five places to eat in New York, mmmmm

2009-03-02T00:00:00-08:00

Five places to eat in New York,

I used to live in New York a good few years ago, and now when friends ofmine are about to go there they sometimes ask me about what to do there. Ikeep trying to think of great places to go in the city, so here is a shortlist of five of my favorite places to eat in New York. Not fancy, notexhaustive, just five places that I remember fondly now that I look back onmy time there. Most of these places are on the upper west side, which iswhere I used to live.

Flor De Mayo

.Chinese Cuban fusion. Best half roast chicken in New York. In fact, besthalf roast chicken on the planet. It comes with a big helping of yellow riceand fried platains. It costs about six or seven dollars and it's on 79th andBroadway, or thereabouts.

Tom's Restaurant(a.k.a Tom's Diner). It has it's own Wikipedia page, anda flickrset. Get the broadway shake. It's a cappuchino choclate malt shake, it's amazing.

Crispy Duck in China Town

Go eat in china town. Get free helpings of green tea. There is a place downthere that has a menu that includes Chairman Mao's favorite dishes. I usedto get the chicken in brown sauce. Most of the places down there are dirtcheap. You can get a good lunch for 5 dollars. I went to one place that dida crispy duck with spring onions and pancakes. It was a bit more expensivethe most of the places, but it was one of the most delicious meals that Ihave ever had. It was called "Peking Duck House" (thanks phillip for reminding me of the name!). Oh,and the green tea ice cream that you can get there is great, and so are thelittle egg cakes, look out for the lady in the booth.

Famiglia Pizza

This is simply the best slice of Pizza in New York, not the biggest, not thefancy schmancyist, just the goddamn best. There are a number of branchesthroughout the city, but the one I used to go to was on 116th and broadway.Incidentally the first pizzeriea in America is also in New York and it'spretty damn good too. It's called Lombardi's

Any Deli for a Sandwich

,But specifically broadway market deli where you should get a Sub filled withroast beef, swiss cheese, mayo, and get it heated, mmmm. It's the archetypalNew York experience. You can get any type of bread and have them stick anytype of anything between the pieces of bread. Bagels are also really reallygood in New York, go to H and H bagels, best bagels in the US, somewherearound 72nd and Broadway.

unknown post

2009-02-18T00:00:00-08:00

I want to find out what all of the ISSN's are for the publications that are printed by Nature Publishing Group. They list all of their publications on the following page http://www.nature.com/siteindex/index.html.

awk '{print "curl http://www.nature.com"$1"index.html | grep ISSN"}' nature-journal-links.txt

Training Diary Catchup

2009-02-18T00:00:00-08:00

OK, so, I've fallen off the wagon in terms of climbing training, or in fact training of any kind.

Last weekend was interesting. I did the introductory PADI course, but the week leading up to the weekend was swamped with work, and all of the nights of this week have filled up too. I think in order to get my running done I am going to have to transition to a morning run a few times a week.

I can give this a try tomorrow and see how it goes. I managed to get my bike road worthy again last night, so that will help with the amount of time I have free in the mornings.

Liowlights:
not finding the time to get any training done.

Hilights:
So there have been some hilights.

- Diving for the first time, even if it was only in the pool.
- Last Monday week ago, nearly onsighting a 6c at the wall. My goal is to tick a 6c some time in March, so that was very promising.
- I tried to get back on the fingerboard again this week, but found it very hard. No power, no motivation. This is actually in the hilight column, as if I can actually get some power back in my hands, given how well I was climbing at the wall recently I should see some improvements!! Let's see how she goes.

Not a smart way to do this.

2009-02-18T00:00:00-08:00

I wanted to find out what all of the

Wrangling data from a mysql db

2009-02-13T00:00:00-08:00

I am working on a project where I have to pass some data from tables in a MySQL DB over to some people who don't use a relational database. The question is, how I can best extract the data in a way that is useable to them?

I've decided to prepare the data for them as XML, as this prevents problems with

My approach uses a hacked together collection of scripts, because, well, as Mark Pilgrim says, because fuck you.

Actually, what I mean is they were just the fastest set of scripts that I could pull together for the task.

It's a mix of perl and python.

1st off, getting the data out of the DB.

Training diary, week 5

2009-02-10T00:00:00-08:00

Saturday 1st
Snowboarding

Friday 7th
Bouldering at the castle

OK, I'm a bit late posting this training update. Last week was the
snowy week in London, and I was lazy, boo me, however after not
climbing for two weeks and given the very poor training week I was
uncertain how things would go, which is to say I thought I would not
be very good, however my expectations were unfounded and I had a
pretty good session. In total 4 5c problems, a 6a problem with an
almost flash on another ( got to the last hold). A couple of weeks off
were not so bad after all.

Highlights:
good bouldering session

Lowlights:
being fuck ass lazy and not getting any running done, shame on me.

tags: bouldering, climbing, training

Converting a mysql schmea into an entity relationship diagram

2009-02-09T00:00:00-08:00

You will need to install XML support as a ScriptingAddition for script editor. You can get this from
http://www.latenightsw.com/freeware/XMLTools2/. Quit and restart script editor after you have installed the extension. I found that when I was trying to compile the code script editor gave me a syntax error before I installed the extension. The syntax error dissapeard after I installed the extension, but the extensions themselves didn't work until I restarted script editor.

http://forums.omnigroup.com/showthread.php?t=1860

Even Santa goes to see the rugby

2009-02-07T00:00:00-08:00

weekly training log, week 4

2009-02-02T00:00:00-08:00

A week snowboarding in Switzerland, 6 days total with a rest day on
the Wed. Pretty fantastic!

I didn't get any running done, and looking at the current weather in
London, it is looking like I won't get any running done this week
either.

My training summary for January 2009 then is as follows:

Number of exercise sessions = 18
Number of days with exercise = 17 (54%)
Total distance = 49 km
Total duration = 54 hrs

Overall the 4 week set at stamina training was not great, I just
didn't do enough. I'm going to extend this one week more into the
first week of February, and the rest of Feb will be back to power
exercises on the fingerboard.

Weekly training report, week 3

2009-02-02T00:00:00-08:00

Saturday 17th

Mile end, decided to try power endurance training, picked a 6a to work
on. 7 leads, and at the end I was still finding it really easy, which
means it was too easy. 6a+ next time. I had a 4 1/2 min rest between
leads. Also did some nice bouldering.

Sunday 18th
ran 12.2 km 80.5 min
did this one http://www.gmap-pedometer.com/?r=2498524

Monday 19th
did some chin ups as a warm up and then 3 sets of semi-dead hangs, lasted
100s, 80s, 60s respectively with about 4 - 7 min rest between.
The last 10 seconds of the first set really fucking started to hurt,
which is good.

Tuesday 20th
4.85 km in 27.5 min, good!

Highlights
getting in a 12 km run.

Lowlights:
Picking an endurance set that was too easy, however I know to pick
something harder next time!

tags: training, climbing

wrapping c code for python using swig on Mac OS X

2009-01-21T00:00:00-08:00

from john d cook
http://www.johndcook.com/blog/2009/01/20/using-swig-to-expose-c-code-to-python/

from
http://www.penzilla.net/tutorials/python/swig/

and combining the two I came up with a way to get this to work on Mac OS X

bash$ cat erf.i

%module erf
#include
double erf(double);

bash$ swig -o erf_wrap.c -python erf.i
bash$ gcc -o erf_wrap.os -c -fPIC -I/usr/include/python2.4 erf_wrap.c
bash$ gcc -o _erf.so -shared erf_wrap.os
bash$ python
>>> from erf import erf
>>> erf(1)
0.84270079294971489

unknown post

2009-01-21T00:00:00-08:00

Code Higlighting in Blogger

2009-01-21T00:00:00-08:00


pbl-ian:python-c root# clear

pbl-ian:python-c root# swig -o erf_wrap.c -python erf.i
pbl-ian:python-c root# gcc -o erf_wrap.os -c -fPIC -I/usr/include/python2.5 erf_wrap.c
pbl-ian:python-c root# ls
erf.i		erf.py		erf_wrap.c	erf_wrap.os
pbl-ian:python-c root# gcc -c erf_wrap.c -I/usr/include/python2.5 -I/usr/lib/python2.5
pbl-ian:python-c root# ls
erf.i		erf.py		erf_wrap.c	erf_wrap.o	erf_wrap.os
pbl-ian:python-c root# python 
Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from erf import erf
Traceback (most recent call last):
  File "", line 1, in 
  File "erf.py", line 7, in 
    import _erf
ImportError: No module named _erf
>>> 
[3]+  Stopped(SIGTSTP)        python
pbl-ian:python-c root# ld -bundle -flat_namespace -undefined suppress -o _erf.so erf_wrap.o
pbl-ian:python-c root# python
Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from erf import erf
>>>

Weekly training report, week 2

2009-01-17T00:00:00-08:00

Friday 9th
2 hours at the castle, top roping with 5kg extra for

Sunday 11th
57 min 8.75 km
http://www.gmap-pedometer.com/?r=2483095

Went bouldering at the Castle, it was good, nothing too heavy, just
tried to do laps on a few nice problems to get tired.

Monday 12th
Westway, lead climbing.
Decided to try some stamina sets, kind of. At least I timed how long
it took to do the routes. 6a+ 3 min bang on. I then tried a 6b++
(wtf?) and that took 2.30 before I was totally shut down, wrong
sequence, too hard a route. I finished the night cruising a 6b, not
bad for the first week back. Did some traversing too, coming in at 4
and 3 min respectively.

Tuesday 13th
Donated a pint of blood, and that kind of whupped my ass for a couple of days.

Friday 16th
7.4 km, 47 min min
http://www.gmap-pedometer.com/?r=2474102

This was a clear 3 min faster over the same route. I was listening to
Cafe Del Mar.

Weekly Overview

Highlights
6b onsight at Westway on Monday.
Getting my milage in by hauling my ass out for a run on Friday night.

Lowlights:
being too tired during the week to do anything on the fingerboard, boo me.

tags: training, climbing

Is RSS dead, and why is it dying?

2009-01-14T00:00:00-08:00

From Science of the invisible I picked up the link to the Read Write Web article on death of enterprise RSS. What is going on here? I'm working in the enterprise and I use RSS a hell of a lot. After trying a number of different strategies I now use Google reader, because it has the fastest way for me to get rid of content in the feeds. Why does the enterprise not get it? I think the issue has to be with signal to noise. RSS opens up feeds to a lot of interesting information, but it is hard work to keep up to date with the content coming in through feeds. Reading feeds is rarely directly relevant to tasks at hand within a company. If we think of RSS as a river of information, most people don't swim in it, but rather go fishing every now and again when they need to find a data point or write a report or something similar. There is a lot of data, but data without insight is of little use.

The tools are not currently in place to easily filter and extract intelligence from that information. Best practices currently involve people creating their own personal workflows, and that is probably a step too far for most people. My take is that what is needed is a tool that can do multiple levels of analysis. Google reader, with it's Trends, is starting to get there, but this only shows reading behavior and does little to highlight interesting things in the content that is being read.

What one wants to extract are probably interesting events or facts where in this context interest has to be a user defined filter. Here are examples of the kind of analysis that might be interesting.

Key word extraction and time evolution of these keywords, either through TFIDF of some other algorithm.

Key words determined through cues from social tagging.

Pattern burst detection.

Feed Feed term correlations and time evolution of these correlations.

Rare but interesting terms (what is not being talked about).

Surprising events, where this can be determined through a Bayseian filter looking for changes from a prior distribution within the feed.

A mechanism for learning based on user preferences.

A nice visualization of all of this.

All of this should be packaged up with filters for doing this analysis on user read items, all items in user subscribed feeds, feeds similar to user subscribed feeds and feeds subscribed or read by the contextual social network of the user, whether that be class mates, friends, or colleagues on a project.

There are many people working on this, for sure.
The Nature Blogs project and Scintilla already do some of these things. Writing this makes me think I should pick up Programing Collective Intelligence again!

Nasty week ahead

2009-01-12T00:00:00-08:00

World War II, Science and Springer Authors

2009-01-09T00:00:00-08:00

Springer have just launched a web app that lets you search for
locations of authors, AuthorMapper. They
have done this by building an XML database with the citation metadata
for all of their papers. You search for terms or authors and it will
generate a google map with the locations for your search terms, or
people. The site also has some other infoporn, including a histogram
for authors published by year for your search term.

This box is along the right hand side of the page. From the default
home page there are two things of interest about this little graph.
This first is the rate of increase in the numbers of authors per year.
From a first rough look, it looks somewhat linear from 1948 through to
1980, and then it starts to speed up a bit. No real surprise there.

The other really interesting thing is the presence of two dips in the
numbers, one numbers, one for the First World War and one for the
Second World War. I've included a screen shot of the graph below
hi-lighting 1945.

I think that this data is historically quite interesting, as Springer
was founded in Germany in 1842 by Julius Springer, and as a result
their archive for those years is probably a good reflection of the
health of intellectual output from Germany during those times.

Springer also played a role in helping to rebuild science in Germany
after the first world war through the publication of some proceedings.
There is a nice letter of thanks from a number of luminaries in the
scientific community to Springer, you can see an image of the letter.

Wekly training report, week 1.

2009-01-08T00:00:00-08:00

Sunday 4th of January
8 km run, in 53 min, average heart rate about 149 bpm.
http://www.gmap-pedometer.com/?r=2468786

Monday 5th of January.
Did a mini endurance set on my fingerboard. I've decided to do a 4
week set of power endurance training, with a view to improving my
sport grade, just 4 weeks and then
I'll switch to finger strength for four weeks. This bascially hurts
and sucks, but hey, that's what you gotta do. I need to be able to
hand from my board for from between
3 to 4 minutes, my best time, as of today, is 90 seconds, let's see if
we can improve that shall we.

Tuesday 6th, 7.4 km, 50.45 min
http://www.gmap-pedometer.com/?r=2474102

Wednesday 7th,
Well, I originally wanted to get down to the climbing wall for my
first session of the new year, but I was a little wrecked, and decided
to have a rest instead.

Weekly review,

Very happy to have gotten 15km total running in my first week back,
and I'm also happy that I've decided to try a four week stint just
doing Power Endurance, either boulder circuits or exercises on the
fingerboard or redpoints. I've never had the discipline to keep one
format of training going for more than about two weeks before. The
goal is also very clear, increase the maximum time that I can hang off
of my fingerboard from 90 seconds to more than 90 seconds. This should
not be too difficult. The ultimate aim with this is to be able to hang
for between three and four minutes, and once I've reached that goal,
I'll start adding more resistance to build up a lot of power in that
time region.

I have decided to recalibrate my training weeks, with day one now
being a Saturday, and I'll be writing up my weekly reports, probably
Saturday morning of the new week, so next update a week on Saturday!

tags: training, climbing

Review of Climbing Goals for 2008.

2008-12-23T00:00:00-08:00

Back in January I set myself some goals for my climbing in 2008. The results at the end of the year were mixed, which means that I succeeded with some of my goals and patently failed with others. The was:

* Onsight Grit E2
* Redpoint f6c+
* 10 weekende+ trips in UK
* 5 red problems in a day in font
* 80 kg weight
* build climbing wall
* get driving license

OK, so, I managed to do the following:

* get driving license

This took a lot longer than I had anticipated. 4 tries sucking up a lot of time an money. In the long term I really believe that this is the most important thing for improving my climbing, so it was worth the sacrifice of time and cash to get it completed. That it took more that half a year longer than I had hoped impacted some of my other goals significantly.

* 5 red problems in a day in font

I over achieved on this one, ticked 8 red problems in one day on my trip to font in September. I had only the one trip planned, and it went really well.

* 80 kg weight

OK, this is a bit of a mixed report. I got down to my desired weight by September, and maintained it close to that level, but in November everything went pear shaped, and I'm back up to a little under 85Kg. The lesson though, is that I can get to that weight. I just need to go running more!

Now, the things I didn't get to do:

* Onsight Grit E2
* 10 weekende+ trips in UK

These two were basically because I didn't have time, money or access to a car. A deluge of weddings (6) absorbed much of my free time. I'm not too bummed about this.

* Redpoint f6c+

Ahh, close, so close. I almost onsighted a 6c indoord. Got to the last hold, fell off trying to hang the last hold. Am confident that a few more trips to the gym would have seen me getting this goal. I actually dropped training for a long swathe in the second part of the year, and only started focussing on this goal towards the end. Earlier focus would have seen me get closer.

* build climbing wall

Again, my fingerboard seemed to be doing OK, and I didn't have the energy to try to build a wall. I will relist this goal for 2009.

So what goals should I set for myself in 2009? My long term goal is to redpoint an 8a by the age of 40. That means I need to improve 7 grades in the next 5 years. I think I'm going to have to get my weight down again, and get my strength up. I'll also have to take into account that I am going to be getting married, and will have some other time commitments in the new year.

Bouldering:

I'll have potentially one trip to Font in 2009, in the Spring. I'm going to set a single goal for this trip, climb font 6a. Preferably climb La Marie Rose. This is a major climbing goal for me, and hopefully by focussing specifically on this I can achieve it. It will also mean lots of bouldering in the new year to up my contact strength, which I actually like.

Trad:

Probably getting any trad climbing done would be good. The number of trips last year totally didn't work out, and if anything the wedding next year will take a bit of time, so let's say 4 trips over the course of the year, with an E2 onsight. I'm also going to say that if I find myself within distance of Quietus then I'll have a pop at that.

Indoor/Sport Climbing:

I need a more refined set of goals. They are going to be:

6c by March

6c+ by July

7a by November

Running:

OK, going running and getting generally fit can really really help, so I'm going to pin my goals at a modest 15km a week. Every week. Except on my honeymoon ;)

Training:

I'm going to keep my training diary up to date on uk climbing.com, but more than that I'll post a weekly update on this blog. And that should be it!

what apps do I want on the iPhone?

2008-12-21T00:00:00-08:00

Well, I just read through the engadget report on the iPhone SDK and it's

looking pretty good. I'm in the middle of downloading the free SDK, mainly

so that I can have a virtual iPhone to play around with, but last night I

got to thinking about what kind of applications I personally would like to

see on the device.

I live in London, but I frequently travel to Europe. I have friends that I

like to keep in contact with in the US as well. Most UK mobile phone

contracts suck for someone whose social network is evenly distributed across

the world. Data rates through T-Mobile when I go the the mainland are

exorbitant, and I only use my N95 as a camera on such occasions cos I can't

afford to check email or look at maps. The two main problems with mobile

tech in Europe at the moment are:

1. No single european-wide tariff for data and calls. As long as the phone

companies get away with making pots of cash it's going to remain this way,

sigh.

2. No free WiFi, it exists in patches, but it's just shit really. One town

in Ireland was recently blocked from rolling out free municipal wifi by the

EU on the grounds that it was anti-competitive (phone companies might loose

revenue). I guess at some point in the future this will change, but for the

time being it sucks monkeys.

So the apps I want to see are:

1. Offline storage of web content, so that I could process, for example,

feeds or emails, while I am in a high danger zone tariff wise.

2. Offline access to a google-maps kind of an interface. Apparently for the

N95 one can download local sheets for areas that you are visiting and browse

them whilst offline, but the software for doing this is PC only. I could set

up a copy of MS on my mac, but frankly I don't have time. I'd like to be

able to dial in a city that I am about to visit, download maps and related

info, and then have that on the go while I am tootling around the place.

Read and post comments |
Send to a friend

test a link

2008-12-21T00:00:00-08:00

test a link guardian

Read and post comments |
Send to a friend

subprime mortgage crises explained using stick figures

2008-12-21T00:00:00-08:00

http://docs.google.com/Present?docid=dtfm9hj_30ccsw79f4

http://docs.google.com/Present?docid=dtfm9hj_30ccsw79f4

This is funny, and a little bit scary too.

Read and post comments |
Send to a friend

star wars

2008-12-21T00:00:00-08:00

"The modified tactical standard missile 3 (SM-3) hit the satellite at an

altitude of 150 miles (247km) while it was travelling at approximately

17,000 miles per hour."

I find this somewhat scary.

Read and post comments |
Send to a friend

social networks

2008-12-21T00:00:00-08:00

There is a tension between the providers of social software, and the

way we behave. When I move from one city to another my social network

changes as that's very location dependent, but when I do have that

network set up for the most part, I don't expect restrictions on where

I can go in that city with my friends. For sure, some friends of mine

might not be caught dead in the palace bar, they only drink in the

stag's head, but I could drop in with my palace friends for a quick

pint and catch up on news.

On the internet distance only affects us on the scale of timezones,

and even there our tail of interaction is much broader. Our changing

activities very much determine the networks we hold on to. I no longer

practice science, but I'm still in contact with my old climbing

buddies. However a big change at the moment is that the places we go

on the internet still don't play well with each other in the same way

that they do in real life.

I hope that truly mobile social networks will emerge, and I think they

will be driven my our address books on our phones. First we will have

real time tracking of the location of our contacts (to the point that

mutual permission is granted), and then this will start to seep into

awareness of location on the web. It's something that has been faces

before, with IM and VOIP walled gardens. So far only email and phone

numbers and physical mail addresses don't have this problem, and

perhaps for that reason those will be the media that crack the problem

first.

Read and post comments |
Send to a friend

scribed is cool, but

2008-12-21T00:00:00-08:00

Scribd has rolled out a very nice

interface to pdf documents. It is neat and slick, but I think what it is

missing is support for getting at data or meta-data in the document. It

would be a great boon to be able to click on a table in a document and have

that tabular data open either in excel or in some online document tool.

Likewise for entities in the document that might have associated metadata.

I'm sure such support is possible and it would turn a nice tool into an

important tool.

Read and post comments |
Send to a friend

sadly I've just discovered that vox sucks

2008-12-21T00:00:00-08:00

I have been liking using vox to blog, mainly because I can email a blog post

from any email client. This significantly reduces the

effort involved in posting a blog post.

I've just discovered that vox has enabled snap previews. I don't like them.

OK, so, some people may like them, and that's OK, but I can't turn them off

on my blog. That's not good.

In addition, when I email a link the link is now broken in vox, so if I type

http://www.google.com you can see that link text but when I embed it in a link the link does not work. boo.

I also like being able to blog pictures directly from flickr, I guess I am

going to have to roll my own solution for a blog again at some point in the

future when I have some time that will allow the bloggy thing to do what I

want.

Ahh well.

tags: blog, suckiness, vox

Read and post comments |
Send to a friend

question of the day, evil vice presidents

2008-12-21T00:00:00-08:00

So the question of the day is if Hillary Clinton becomes

Vice-President, would she be more or less evil than Dick Cheney?

Now Dick Cheney is evil Inc. but the guy has never really pretended to

be anything that he isn't, whereas Hillary would obviously

lie and make policy in any way possible to win votes, which means that

though she might not be as intrinsically as evil as Dick Cheney, she

might make as bad a VP as him.

I think that she probably wouldn't be as actively evil as Dick Cheney,

but sadly she is beginning to point in that direction.

Read and post comments |
Send to a friend

progress

2008-12-21T00:00:00-08:00

If you are interested in the progress of my bet you can follow here:

hmm, my embedded image is not showing up here, shame, well, the link is here: http://www.bellygraph.com/graph/view/300

Read and post comments |
Send to a friend

perl and again,

2008-12-21T00:00:00-08:00

Ever since Ian Clarke mentioned the potential delights of python to me

whilst passing me in a corridor back in 1999 I've been tinkering around with

that language.

Christ, that's almost 10 years. I like python, it gave a great sense of

freedom after spending a heavy five years working with various flavours of

FORTRAN, from

FORTRAN 90, through to High Performance Fortran. Well, now here I am

tackling perl. It's a gem of a language, but by christ it's scary. The power

that it gives you

in it's ability to interpret so freely is also it's danger.

From the end of the introductory chapter from Programming Perl we have:

"But in the end, you must create you own view of Perl. It's your privilege

as an artist to inflict the pain of creativity on yourself. We can teach you

how we paint, but we can't teach you how your paint. There's More Than One

Way To Do It. Have the appropriate amount of fun."

For writing code at the speed of thought I can see how this is going to be a

beguilling language, however for large structured projects it worries me

that this language is going to slip out and away from the instantly

comprehensible. Perlhaps that's not the case, I guess I'll just have to find

out.

Read and post comments |
Send to a friend

nice idea for improving the six nations rugby tournament

2008-12-21T00:00:00-08:00

Shaun Edwards <a href="http://blogs.guardian.co.uk/sport/2008/02/22/bonus_is_the_carrot_that_leads.html">writes</a> in The Guardian today
that introducing the bonus points system to the Six Nations rugby tournament could help make it more entertaining (not that it needs much help in that regard, as it is a fantastic tournament as it is). I wholeheartedly agree. At the moment you get 2 points if you win a game, and 1 point if you draw. Under the bonus system you get an extra point if you score 4 tries and you get a bonus point if you are a losing team and you hold you opposition to a gap of 7 points or less. I think you also get more points for a win. I have only one improvement on this formula. You should get a bonus point if you are wearing green. I fear that this last rule enhancement is likely to be the only way Ireland will win the championship in the near future.

Read and post comments |
Send to a friend

networks; head to head

2008-12-21T00:00:00-08:00

The second talk in the evening CREEN parallel session is from J. Holyst

about phase transitions in coupled complex networks where the network

properties are different in the two groups.

In one group there are a smaller number of members, but they are tightly

coupled. The second group is larger, but less tightly coupled. The model

uses an Ising model. The less coupled group has higher fluctuation in

opinion, and when it is brought into contact with the smaller group the

larger group undergoes a phase transition. To reverse the process if you

can find a hub in the tightly coupled system then when you convince them

their opinions gain traction.

This is a really nice paper and there are obvious tendancies to draw

parallels with real communities, such as for example two scientific

communities, or an immigrant community, but I am fairly certain that

this would at the moment be too simplistic.

He goes on to show a face off between ER graphs vs BA graphs, and this

shows that network structure has signifigant effect on the suseptability

of the community to opinion change.

One big application could be to look at these results in the context of

the new vote that Ireland will probably undergo early next year to

re-ratify the Lison treaty (I mean, I assume that is what is going to

happen in Ireland next year).

Read and post comments |
Send to a friend

network branching, netsci08

2008-12-21T00:00:00-08:00

I'm really torn by the number of great talks on today. There are three

parallel sessions, and for each time slot I want to be in at least two

places at once. I'm going to try to pick out talks that have some

relation to online social networks, community detection and scientific

networks, but some of the talks on the theory of clustering are

conflicting directly with some use cases of looking at some online

social networks. Ahh, what a dilemma.

The opening talk of this session was from Stwphen Uzzo talking about the

The next talk was by M.C Gonzales looking at the network of travel

patterns. This was the paper that made the cover of Nature.

The big question is trying to find out what the travel patterns of

people are. Thhe big problem is that getting data is apparently quite

hard.

The solution is to follow mobile phone signals, following 10^5 people over

10^6 locations over six months.

I'm looking at the movie of their data, and it is clear that many people

don't move very much, and other people move a lot. Of course one wants

to know some information about the people to see what effect like age,

wealth and occupation will have on these results. Again I'm looking for

something surprising.

There is a nice graph showing corellation over time, it is hugly spiked

on 24 hours. Not surprising, but a good reality check on the data.

I'm going to head to one of the other sessions after this talk.

This talk, though, is very nice. Once again there is evedence that our

behaviour is depressingly regular. Also the longer a journey the more likely

that a journey is going to be linear.

Read and post comments |
Send to a friend

netsci08 opening keynote

2008-12-21T00:00:00-08:00

Nicholas Christakis, Harvard, "eat drink and be merry, the spread of

health phenomena in social networks".

This talk is looking at the spread of desies throgh social interactions,

rather than other types of interactins. The main study was looking at

obesity using the Framingham Heart Study Social Network. This seems like

a very famouse social network health related study, so I'm not going to

go into detail about that, but the bottom line is that they were able to

construct the social interactions from this study by digging through the

huge paper archive. They were able to look at friend, relative and

co-worker ties.

The main study was looking at about 5k individuals out of 12k, taken

from 1973 onwards.

Nice, node ssize is related to a person's weight!

There is clear clustering of obese nodes in the network, now is this

clustering random or structured?

Well, it's more clustered than random.

There are a couple of reasons why this might be the case. It could be

that obese people like each other, people might be susceptible to local

factors, or there might be some kind of peer pressure.

By looking at time evolution the hope is that they might be able to find

'patient 0' for the obesity epedmic. OK, video is coming up now ...!

OK, looking at people getting fatter all over america from 1972 onwards,

I'm going for a run later!

The effect is not centered in one location, but it seems that it's an

epidemic that had multiple starting points in the network.

Looking at the directionality of ties of friendship helps you make

inferences about causes. Wow, if you are friends with someone who is

friends with you, and they get obese, you have 300% greater chance to

gain weight. Stay friends with thin people!

It looks like much of this is driven by social norms.

They also have gwo data from the network, that is really cool.

They can convert location to wealth, and can take this into account when

looking at the evolution of the network.

This data is really really cool.

No drop off in effect with distance, it is really the social tie that is

important.

They also looked at the effect of smoking, and were able to take this

into account.

So their working hyppothesis is that it might be the spread of behaviour

and habit, perticluarly shared behaviour, going runnning vs going for a

beer.

It might be the spread of an idea, the spread of what an acceptable body

size might be.

OK, that's pretty amazing, and you can tease a hugh amount of

information out of this study. Liklihood of quitting smoking, of how

that is effected by education, and friendship tie.

I have to say, there is not a lot of results that are amazingly

astonishing. They have food networks, like the bannana network and the

friend chicken network.

They are also looking at emotions. We know that emotions can spread

through groups, on diads. Could emotions spread hyper-diadically, and

over longer time frames?

There is strong clustering of happienss, and your happiness seems

coreelated with people who are outside of your direct social horizon.

Interestingly happiness does not spread in the workplace (I think that

was the point), but happy people have higher clustering and better

centrality in the network.

There seems to be a half life for catching happiness from your network,

this is about 6 months. There is also a strong local effect, you need

happy people to be within about two miles of you, and to be having happy

events happening to them every six months or so.

Ahh, you can look at smiling on facebook. Right, I gotta put up some

happy pictures on my profiles!

Ahh, thiness also spreads, but the reason they have been looking at

obesity is that this study is looking at the obesity epidemic. The

network shows you the magnification of the phenomena, not the cause or

origin of the phonomenon.

Interesting question, if you wanted to hire flight attendants who you

didn't want to gain weight, should you hire them based on the bmi of

their friends? Well, the answer is that in a workplace if a certain

behaviour begins to spread it is likely to have a network effect. The

flpiside is that you can use these network effects to more economic

effect by trying to promote certain behaviour through targeting core

groups in the workplace.

Read and post comments |
Send to a friend

netsci08 blogging

2008-12-21T00:00:00-08:00

I'm in Norwich all this week attending Netsci08

http://www.ifr.ac.uk/netsci08/, the internatinal workshop and conference

on network science. It's a week long event, and broadly speaking it

looks like there are three types themes that are being discussed here:

biological networks, pure networks science and community detection in

networks, principlaly emergent networks of the kind we see in the

internet.

I'm twittering about the meeting using the tag #netsci08, but it seems

that I'm the only one out there in the twitterverse who is also at this

meeting. Not enough power in the lecture hall, and wifi is a little

ropey, but the conference is pretty good so far.

The talks on Monday were about some basics on network mathematics, and

on network science in the social sciences. I'll go back over my notes

and give a quick report on them when I get a chance to catchup, but the

discussion in the evening was pretty interesting, and the talk in the

morning touced on some very important topics.

The Tuesday morning tutorial is on economics and networks. The morning

model was very simple, and I think that's fair enough, but I got the

feeling that the level of the audience, at least on the side of the room

that I am sitting on, was high enough to have taken a bit more robust

model, so I got the feeling that there was some discomfot with the model

presented.

The after-coffee section is focussing on social influencers, now this is

interesting.

How is it that information flow is highly assymetric in the world?

The model is a mmulti-state model with differeing outcomes. Individuals

don't know the true state of the system, but they have beliefs about the

states. Sounds like a hidden markov model.

The model is stationary, and we want to see how the choices we make

change the beliefs that we have. Could be a bayseian network? Let's see.

What I am hoping to see from this model is how reccomendations can

travel throgh a network. There is a network of communication between the

network. The model can integrate dynamics, the dynamics of belief.

There is also feedback between actions and beliefs. The main result is

that as time goes by new information has less effect, and so beliefs

converge in the network. This is a consequence of Martingale's theorm.

The big question is whether we get optimal actions, and the big result

is that the ability to explore the action space and find the best action

is depenant upon the structure of the network. That is really

interesting.

Oh my God, someone has an OLPC machine in the audience, how cool is

that!!

Anyway, back to the talk. So this is indeed a Bayseian network. The

anti-intutive outcome from this model is that if you have to build a one

time only network that can't be changed later, then you have the best

chance of getting optimal behaviour if no one person has undue

influence, hoever I think that for online social networks there is a lot

of dunamics going on that can pull you out of local sub-optimal minima.

Read and post comments |
Send to a friend

mozy backup service

2008-12-21T00:00:00-08:00

I've been trying the free account this week. I've backed up 1.6 GB of data.

It took, with given interruptions of a normal work day, about 4 days to

backup this amount of Data.

I am beginning to think that the world of offline remote backups is still a

little bit away!

Read and post comments |
Send to a friend

incentives

2008-12-21T00:00:00-08:00

So after reading about some economists who decided to use economic theory to

loose weight I have decided to follow suit. They both decided to loose a

certain amount of weight by a certain time and places $5000 in a bond.

Whoever didn't make the weight lost their portion of the money to the other

person. The idea was to affect behavior through an incentive that

was significant. Well, I'm not a well paid economist, but I have decided to

follow their example.

A couple of years ago, back in 2004, my weight had leveled out at 80kg. I

kept a daily record as part of a climbing training log, and it actually

varied over the year between 77.4 and 80.6 depending on how much climbing

i'd been doing.

Then I moved to The Netherlands and within 18 months had put on 15 kg.

Welll, I'm back down to 85, but those last 5kg have been hell to get rid of,

so tonight I made a bet with my girlfriend. If I don't lose those 5kg within

5 months then I pay her 400 british pounds sterling. That gives me until the

1st of August.

I reckon I need to run/walk about 100 kms a month to hit the target. I'm

already walking 3km a day to work, and 3 back again. If I were to walk to

work every day that would already be close to 120 km, so it's within the

realm of possibility. Even more so if I can stop having those sneaky beers

in the evening.

Spurred on by this new motivation I just went for a 6.6 km run, phew.

Read and post comments |
Send to a friend

ikea is like a biological machine

2008-12-21T00:00:00-08:00

We are just unpacking some more furniture and we have discovered that the

spacers inside the boxes were made up of cut up pieces of IKEA furniture,

probably items that had been returned. It reminded my of the cells ability

to recycle old components.

Read and post comments |
Send to a friend

iPhone prediction market

2008-12-21T00:00:00-08:00

Well, a prediction about a new market that will spring up (perhaps it

already has) around the iPhone. So it costs $99 dollars to get an app that

you have built onto the phone through the App Store. If I design a vanity

app I'm not going to want to pay that much, but I might pay 5, 10 or even 15

dollars to see the child of my creativity bouncing around on the little

device. Some dev shop is going to offer to use their app key to load other

people's apps to the App store for sure.

Read and post comments |
Send to a friend

how to learn python

2008-12-21T00:00:00-08:00

In response to a query that popped up on Friend Feed:

I taught myself years ago with O'Reilly's Learning Python. The web

book 'Dive into Python' is essential, and the blog posts on

respectively 'How to Code like a Pythonista' and 'How to think like a

Pythonista' will get you a long way towards familiarizing yourself

with common idoms in the language.

Read and post comments |
Send to a friend

getting married and buying an iPhone are probably incompatible.

2008-12-21T00:00:00-08:00

Having just gotten engaged and begun to look at the relative costs

involved in organising a wedding I woke up this morning with the

unavoidable realisation that the need to save for the wedding is going

to make purchasing an iPhone, while I have a perfectly normal

n95 that works rather well (especially since the broken key unstuck

and started working again) untenable. My cunning plan had been

to wait until the end of my current contract, at which point the new

iPhone would be available, but the need to actually pay for the device

in addition with my desire to invite rather a lot of people to my

wedding means that I ought rather probably wait until after the

wedding before

I think about getting a new flashy gadget.

Read and post comments |
Send to a friend

further confustion

2008-12-21T00:00:00-08:00

So, after a little more investigation it is clear that earlier links are

broken and new links have the snap preview.

Well, I guess I am just going to have to write links as plain text until

they get this thing sorted out.

tags: vox sucks

Read and post comments |
Send to a friend

drowning in the streamosphere

2008-12-21T00:00:00-08:00

I've been thinking about the streamosphere (as coined by Euan Adie),

and how I am starting to get dragged down in it again. I'm getting

daily notifications from pounce, twitter, friendfeed, linked in (I've

been ignoring facebook for ages, facebook, you bite my ass). What I

really need is a way to manage my notifications and communications in

the way that I now manage my rss feeds, through a bespoke piece of

aggregation kit.

Read and post comments |
Send to a friend

digital archeology, finding traces in the bits

2008-12-21T00:00:00-08:00

This

article in Guardian today is describing a method of restoring colour to

some original BBC tv shows whose master tapes were wiped in a purge of the

BBC archives.

The shows were originally made in colour, then the master tapes were

destroyed to make more room in the BBC archive, however black and white

copies were made for distribution to countries which didn't have colour TV

capability. The black and white versions were on a smaller format (16 mm)

and of a lower resolution.

The method of recording was the following, the colour shows were displayed

on a large screen and this was re-recorded onto the 16mm tape.

During this process owing to an artifact from the colour, on the black and

white film there is a speckled fingeing error.

It describes this problem in the article as follows: "However, there is a

more relevant problem. Any black and white telerecording of a colour

programme is prone to pick up interference from the colour encoded video

signal. This manifests itself as a pattern of small grey dots, called

chroma-dots, across the picture."

A filter used in the re-recording process could eliminate this error, but at

the time of the recordings the error was considered so minor that often the

filter was omitted. Back then TV was throwaway and space in a basement was

considered more valuable than any idea of cultural heritage. Now we look

back at the decision to scrap the original recordings as somwhat akin to the

burning of the library of Alexandria. Storage, especially digital, is

bountiful.

However, owing to what was then considered to be noise, an error, an

imperfection in the process, James Insell has devised a technique to map

back from these chroma-dots to the original colour. And so by looking at the

finest structure of the current artifact through HD recordings of the 16mm

film we can recapture a state of the past that might have been lost to us.

Digging in the digital detritus has uncovered gold.

The formats of our cultural heritage are changing, and there is ever the

danger that we might loose large chunks of our past when the ability read a

certain format disappears. For specialist cases, such as the BBC archive,

there will be, for some time to come, the likes of James Insell who will be

willing to do the archeology needed to reconstruct from the remnants, but

looking at the growing sprawl of media in my home I have a number of digital

cards that can't be read any more, a zip drive that may or may not contain

code from a summer of astrophysics, and some very personal movies on

betamax, that lie in their boxes, becoming more tomb-like as the years roll

by.

Aside from the personal onus to maintain my own history, I was really

excited by this guardian article. I loved the idea of recovery from

the minutia. It is an interesting example for the need for losses

compression techniques.

Read and post comments |
Send to a friend

collaborative intelligence and the pidgeonhole problem

2008-12-21T00:00:00-08:00

O'Reilly are hosting a Collaborative Intelligence foo camp this coming

weekend. There is some discussion bubbling out already on crowdvine at a group that has been created for this

meeting.

One of the participants, Greg Linden, says the following:

"Hi, Cass. Absolutely, this is called the pigeonhole problem and can be an

issue if personalization is done poorly.

Done right, personalization enhances discover by helping you find things you

could not easily have found on your own. Discovery in vast quantities of

data is what personalization is designed to do. The key is to make sure the

personalization reaches beyond the obvious and into the surprising. If you

do that, personalization reveals the full breadth of the data and enhances

serendipity."

and then

"Hi, Cass. Sure, I think there are things that can be done. Most of this

depends on the technique used for personalization. For example, if you

implement recommendations by showing people items that match on keywords or

from the same fine-grained subject category, you tend to get nearly

identical items and little diversity. If you use user behavior, such as what

people tend to buy together, you can often get more interesting patterns

out, especially if you tune the system to try to reach further afield (at

the risk of more spurious recommendations).

It's a great question and one that troubles all of us when thinking about

personalization. We are filtering, but trying to do it in a way that focuses

attention on interesting things, not that limits what people see.

I look forward to talking with you more about this at the conference!"

It brings to mind the idea that there exists a landscape in the search

space, and what one wants to do is avoid local minima, but explore enough of

the space to find related results that are not directly in

the neighborhood of the item of concern. This means there must be a

correspondence to the temperature of the system. Getting out of local minima

can be achieved through an annealing process.

Balancing relevance and serendipity through some

constrained minimization process, like the one set out in the information

bottleneck. How much better performance will you get out of these approaches

over simple algorithms and what will be the relative computational cost?

Does anyone know of any literature about this?

Read and post comments |
Send to a friend

climbing milestone

2008-12-21T00:00:00-08:00

Last night I got my first UK 6a boulder problem at the Castle climbing wall,

it's been a while since I was making progress with my climbing, so that was

great.My climbing diary is here: http://jottit.com/h94hx/

Read and post comments |
Send to a friend

chasing cars

2008-12-21T00:00:00-08:00

This morning as I was leaving Shoreditch park a guy in a black masarti

with the top down passed my on my bike.

I continued along the canal as usual and as I came past sainsbury's in

islington saw the same guy in the same car, as I passed him!

Bike 1, Masarati + London Traffic 0.

Read and post comments |
Send to a friend

bus podcast idea

2008-12-21T00:00:00-08:00

I had a great idea yesterday while I was engaged on a personal odyssey on

London buses. I was on the top deck of, I think it was the 176, heading down

past Old Street into the square mile, round St Paul's (with a quick glimpse

of millennium bridge, and then on into the strand. The bus went past so many

statues, churches and historical buildings, but I knew only a few of them.

It struck me that a podcast that could be used as an audio guide for great

bus routes. You could chapter the audio guide by street so that it was easy

to navigate according to different traffic conditions.

The Dublin Tourist Office does something like this already for walks around

the city

http://www.visitdublin.com/multimedia/DublinPodcasts/iwalk.aspx?id=275,

but with London, or other big cities, providing same for bus routes seems

like a good idea.

tags: podcast, idea, cool, london, tourism.

Read and post comments |
Send to a friend

blogging darwin

2008-12-21T00:00:00-08:00

One of my friends is writing a series of blogs about Darwin.

It looks like a very nice series of blog posts, and I look forward to the

rest.

You can read it here.

Of course as a physicist I disagree with one comment in this first post:

"From an intellectual point of view, the Origin is the most significant work

of the millennium, and the most important work of non-fiction ever."

I would vote the atomic hypothesis into that position, with, in addion, a

nod to deep time and the extragalactic distance scale, but we each have our

own scientific peccadilloes so I won't argue too much on the point.

Read and post comments |
Send to a friend

a great presentation about books and community

2008-12-21T00:00:00-08:00

My colleague Gavin Bell just uploaded a great presentation about creating communities of readers out of readers of books. You can see his presentation <a href="http://www.slideshare.net/gavin/gavin-bell-from-readers-of-books-to-a-community-of-readers-oreilly-toc08/">here on slideshare</a>. He gave this talk at the Tools Of Change conference in New York earlier this month.

Read and post comments |
Send to a friend

Your Brightkite Invite

2008-12-21T00:00:00-08:00

I just got the following Yay!

"Hi,

Somebody has sent you a Brightkite invite. Brightkite is a

location-based social network, currently in private beta.

Personal message:

The wait is over! Here's your Brightkite beta invite. Enjoy!"

Then on sign up discovered that it is only supported by a number of US

carriers, boo.

Read and post comments |
Send to a friend

Why the LHC is not really that impressive

2008-12-21T00:00:00-08:00

Yesterday the Guardian had a special pull out section dedicated to the

LHC. If you browse through the articles you find lots of comments along

the lines of

"temple to mystery and imagination", "a journey to the edge of

understanding". "a modern cathedral to our relationship with the

universe", and so on. From the superlatives that are being written one

would think that the LHC is the best thing to happen to enlightenment

since some fat chinese guy sat beneath a tree, and that it is the summit

of human imagination, achievement and art. Well, I just don't buy all

of that crap.

Reading these articles got me thinking about what the LHC is, and

fundamentally it's just a larger detector than what we already had

before. As I see it, it's an inevitable extension of what you do if you

want to measure something that we already know how to measure (particle

tracks), with better precision over a higher energy range.

The bottom line is that we have been doing this since the 1920's.

If you look at it as just being an artifact then it is neat, but there

are many other piece of artistry that required as much imagination,

effort, skill and chutzpah to bring together. The moon landings are one,

the regularity of probes landing on mars another. The engineering

required to make a large city like New York work always blows my mind,

and that emerged from a bottom up self organization of 15 million souls

trying to find a way to survive in an area of land a little too small

for them all.

As we look around the world at the things we as a species have built

there are many such artifacts that can inspire our awe and wonder. I

don't think that the LHC can lay a claim to be at the pinnacle, though

no doubt it is a good example of a big complicated object that make

people look small when they stand beside it.

There is something to wonder at in all of this, and that is the idea

behind the inevitability of something like the LHC. That idea is the

atomic and quantum electrodynamical nature of the world. In that there

is something to be proud of as a species. I don't see the LHC as being a

radical departure from this idea, but rather an object whose existence

is quintessentially rooted in that idea.

One could almost argue that the LHC represents a failure of the

imagination. We are faced with limits to our ability to test the

mathematics that we have written down against the atoms that we write

with. We cannot tease apart the Fynemann diagrams to tell us more about

the world, and so we resort to a bigger hammer rather than a more subtle

approach that might look to other ways to coax the mysteries of the

universe out of their hiding places.

There have been some papers that have come out recently looking for

connections in the physics of super fluids with the imagined state of

the early universe, the idea being that looking at the behavior of

vortices in super cooled liquids could demonstrate identical physics to

the phases of matter at the point of various decouplingings in energy

scales. It's pretty clear that these models are yet toy models, but

perhaps they point out an orthogonal direction to building massive atom

smashers.

I want to be clear that I do applaud the work of the thousands of people

working at Cern, and I do think that the billions of euro that something

like this costs is more than worth the investment. I appreciate how hard

it is to deal with systematics on something of this scale, and it is a

minor miracle, but I just don't think that the artifact deserves unconstrained

adulation over the ideas that is reflects.

Read and post comments |
Send to a friend

We have been nominated for a webby

2008-12-21T00:00:00-08:00

Nature, the company I work for, has been nominated for a Webby, woot!.

OK, so we are up against Nasa's earth observatory, and some other

great sites in the science field, but it's really nice to get a nod

and see that some other people out there like the kind of work that we

are doing. If you want to go and vote you can at this link herehttp://pv.webbyawards.com/ballot/home/1.

Read and post comments

|

Send to a friend

Upcoming.org Founder Creates Fireball (Fire Eagle + DodgeBall + Twitter)

2008-12-21T00:00:00-08:00

Ahh, convergence, ahh, the ability to pinpoint your friends on your

mobile phone. I've been waiting for this to happen.

Sent to you by Ian Mulvany via Google Reader: Upcoming.org Founder

Creates Fireball (Fire Eagle + DodgeBall + Twitter) via TechCrunch by

Erick Schonfeld on 4/22/08

Remember DodgeBall, the early social mobile network that languished

after Google bought it? So does Leonard Lin, a founding member of

Upcoming.org who recently left Yahoo, where he organized Hack Days. He

helped write the code for FireBall, a clever mobile geo-location app

that brings back the promise of DodgeBall using only other existing

services with public APIs.

FireBall is a way for people to keep track of where their friends are

on your mobile phone. It uses Yahoo’s Fire Eagle as a geo-location

broker and Twitter. It is basically mashup of the two services, plus

some functionality from Upcoming.org. People add all of their contacts

on Twitter and authorize Fire Eagle to share their location with

Fireball. “Instead of creating a new service that forces you to add all

of your friends,” says Lin, “we end up using Twitter for messaging,”

When you want to find out where your friends are who have also signed

up for FireBall, you send a message to a Fireball account on Twitter.

You get back a text message with a Tiny URL link. When you click on the

link, it opens up a KML file that launches Google Maps on your cell

phone and hows you all your Twitter friends as pinpoints on the map. So

your Twitter contacts serve as your mobile social network. You can also

Twitter in your location. Simply mention a room at a conference, for

instance, and it can pinpoint exactly where you are through integration

with Upcoming.org,

FireBall launches today in a private beta for attendees to the Web 2.0

Expo. The first 100 TechCrunch readers who are attending Web 2.0 Expo

and send an email to “Fireballme+TechCrunch [at] gmail [dot] com” will

recieive an invite. Right now, the service only works in San Francisco.

CrunchBase Information Fire Eagle Twitter Upcoming Information provided

by CrunchBase

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered

Daily.

Things you can do from here:

- Subscribe to TechCrunch using Google Reader

- Get started using Google Reader to easily keep up with all your

favorite sites

Read and post comments

|

Send to a friend

Ubigraph

2008-12-21T00:00:00-08:00

Ubigraph (http://ubietylab.net/ubigraph/) Ubigraph (sorry for the

fucked up formatting, I think my blog host vox is still being shitty

at recognising simple html formatting in input, mixed with line

breaks, it's just a frickin url link for god's sake) is a nicely

implemented engine for making 3-D graphs and plots. I installed it and

got the default python interface up and running in about 5 minutes. My

initial reaction was "this is cool". It runs on a server that you

communicate with using an XML-RPC interface. Then I had a moment and

realised that I couldn't think of anything, other than trying to plot

graph relationships in citations, to do with it. I want to look at the

graphing of these relationships, but need to find the time to mine

some data first, so I'll just have to put this on the shelf for a

moment. Before I do, I wanted to spin off a quick blog post to remind

myself that this really was very easy to work with. You just create a

graph object G. one could use this in conjunction with NetworkX and subclass the

ubigraph object from the networkx object and as you wrangled you

networkx object you would see it appear in ubigraph, that would be

cool.

Read and post comments |
Send to a friend

The gap between Open Access and Editorial Quality

2008-12-21T00:00:00-08:00

I'm reading a great article by Joe Esposito on Open Access. You can read

the article

here:http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;cc=jep;rgn=main;view=text;idno=3336451.0011.203.

The article's main claim is that attention is the barrier to

disemination rather than access. Another claim of this article is that

paid for publishing platforms are good for readers in that they act as

an attention filter.

One of the main claims of the article is that Open Content in a sea of

other content will exist beyond the scope of peer review and editorial

review. That is certainly the case at the moment for the most part.

Beyond Google's search rank, there are few other effective ways of using

machines to rank the quality of content, and certainly none at the

moment that apply specifically to academic content.

I think that one of the issues is that the volume of signals about the

quality of data that can be data mined compared to the data that needs

to be mined is still tiny, and the tools to do this mining almost non

existant. Social signals from tools such as Connotea are only just

beginnig to create traces, but they are still small.

There is a hope that automatic data mining of sources will push out the

need for manual editorial control, and there are platforms out there

with enough computational power to do this, it's just that these

platforms at the moment have much more important jobs to do.

Even when hardware and storage really are commodotized (and the

existence of S3, EC2, AppEngine should leave no doubt in our minds that

this is the way things are going), who is going to write the algorithms

to review papers for us. Perhaps we only need a few people to do this,

and so I shouldn't worry too much about that, but I still don't see this

kind of thing happening for a few years yet.

There is another interesting aspect of Open Access that springs to

mind while reading this paper.

The article does not address the machine redability of published work,

and how restrictions on that type of access can hinder auto-mining of

facts and figures, the aggregate results of which may command

signifigantly more attention than the individual componenets.

Finally Joe has a nice little analysis of the kinds of costs associated

with building new publshing platforms to market.

Alltogether I really liked this paper.

Read and post comments |
Send to a friend

Testing out a new blog platform

2008-12-21T00:00:00-08:00

So, I've been having some problems with the Vox service for some time,

and now I have a twine invitation, so I am going to test out Twine as

a platform for blogging.

The way I blog is that I tend to write up my blog posts in a mail

client and then mail these to the blog engine, but so far Vox as a

service has been soso.

For the next while I am going to try posting the same content to both

locations and I'll see which one I like most. Ultimately I want to

mirror this content on my own domain/site.

One of the problems that it has been having is mis-rendering html,

let's see if I can get a link to my home page working in this post,

mulvany.net

Read and post comments

|

Send to a friend

Soft peer review

2008-12-21T00:00:00-08:00

The following paper "Soft peer review: Social software and distributed

scientific evaluation" was passed along to me by alf today. I think

another copy has been haunting my file system for a few days, but this

seemed like a good reason to sit down again with it.

It's by Dario

Taraborelli and the abstract is as follows:

Abstract: The debate on the prospects of peer-review in the Internet age and the

increasing criticism leveled against the dominant role of impact factor

indicators are calling for new measurable criteria to assess

scientific quality.

Usage-based metrics offer a new avenue to scientific quality assessment but

face the same risks as first generation search engines that used unreliable

metrics (such as raw traffic data) to estimate content quality. In

this article I

analyze the contribution that social bookmarking systems can provide to the

problem of usage-based metrics for scientific evaluation. I suggest that

collaboratively aggregated metadata may help fill the gap between traditional

citation-based criteria and raw usage factors. I submit that bottom-up,

distributed evaluation models such as those afforded by social bookmarking

will challenge more traditional quality assessment models in terms of coverage,

efficiency and scalability. Services aggregating user-related quality indicators

for online scientific content will come to occupy a key function in

the scholarly

communication system

and I get a mention in the acknowledgments, which is cool.

It is a very nice essay on the potential of social bookmarking as a tool for ran

king academic articles, in addition to adding metadata to scientific articles. D

ario discusses the issue of ranking the expertese of people who are bookmarking

and proposes a really nice method to get over the scaling problem that is inherr

ent when we try to intoduce manual methods to rank people. He suggestes that a u

sers notes and annotations could be made available about a bookmark on an anonom

ous basis. Others would have the option to copy these annotations, or rate them.

This would be a form of soft peer review on the annotations, which would in tur

n effect the standing of the person creating these annotations.

There would be ways to cheat this system, but with enough signal, one hopes that

such noise could be drowned out.

The paper also pointed out http://www.naboj.com/ which I'd not

seen before and which is

pretty amazing.

I really like this paper. Thanks Dario!

Read and post comments |
Send to a friend

Predictably Irrational

2008-12-21T00:00:00-08:00

I was in, of all places, Godalming, at the weekend and ended up

browsing in a book store for a few moments. I saw what looked like a

very interesting book,
href="http://www.amazon.co.uk/Predictably-Irrational-Hidden-Forces-Decisions/dp/0007256523/ref=sr_1_1?ie=UTF8&s=books&qid=1211186399&sr=8-1">Predictably

Irrational. The book looks at the processes behind bad or

irrational decisions. The author is an economist and it seems the aim

of the book is to help us to see the emotional effects that influence

our decisions, and that lead us to poor decision making. This is a

theme that is dear to my heart, as it is closely related to the way

that science policy gets determined, insofar as the facts about

specific scientific domains are often swamped by emotional reactions

to what people think the science is about.

I didn't buy the book yesterday, as I have a very large current

reading list, but not wanting to feel that I was costing myself some

opportunity I actually remembered the name of the book. I was

delighted this morning when I found that the author has a great
href="http://www.predictablyirrational.com/">site about the book

which includes a
href="http://www.predictablyirrational.com/?page_id=17">blog. The

author Dan Ariely also has a
href="http://en.wikipedia.org/wiki/Dan_Ariely">wikipedia page.

From the site and the blog there are links to some papers that seem to

form the basis of the book, so now I can get some small chunks to read

to satisfy my curiosity about the subject.

Read and post comments |
Send to a friend

Networks in Space, Mark Newman, netsci08

2008-12-21T00:00:00-08:00

Mark Newman, Networks in Space,

This is about networks in geographic space.

Mark is looking at properties of networks that are tied to geography.

Transport networks are a good example, and we are looking at the

difference between road and air networks.

The road and air networks are very different, even though you use both

of them for getting from A to B.

There is different bahviour, could I say 'driving' the use of these

networks. For roads we want to minimze the length of our journey, but

that's not such an important factor in flight journeys. When we fly we

like to take direct flights, and minimize the number of flight hops that

we take.

If you model this behviour you get out networks that look a lot like

road and flight networks.

Their first model looked at connecting randomly distributed nodes.

In order not to get influenced by population density they made a map

that is rescaled by population density. This is called a cartogram.

You can see some really nice election cartograms that Newmann and

Gastern made here: http://www.cscs.umich.edu/~crshalizi/election/

There is a really nice historical example from Raisz from the

Geographical REview from 1943.

It looks like most recent attempts have been hand-drawn, but they look

pretty shit.

Newman and Gastner made a difffusion algorithm that allows you to do

this quickly.

OK, it looks like this started off as a network talk, but segwayed into

a demo of this mapping technique. Ahh, no, we are back to looking at

airports.

The interesting result from this talk is that the best covering fro

utilities such as airports or post offices does not grow lineraly with

popultaion, but to the power of 2/3.

This was also a pretty nice talk.

Read and post comments |
Send to a friend

Max OS X does not have gethrtime and as a result cannot load Time::HR

2008-12-21T00:00:00-08:00

I just discovered this after poking around for an inordinate amount of time.

The trick is to use Time::HiRes instead.

Now someone should write a bridge function that overloads the Time::HR

functions with Time::HiRes functions which would make a code change as easy

as loading the overload module.

Read and post comments |
Send to a friend

I didn't expect lock in so quickly

2008-12-21T00:00:00-08:00

I've just signed up for an account with twidox, which is a start up

that is collecting shared documents of interest to scientists, I

believe. They are in the private beta stage, but had a link on their

homepage for requesting an account. I got the following message when I

hit the verification link:

Registration is taking place!

Many thanks for you interest in twidox. Your account has been activated.

We will send you your private-beta lock-in details very soon.

We thank you for your support.

Your twidox-team

Read and post comments |
Send to a friend

How I learned to stop worrying and love RSS

2008-12-21T00:00:00-08:00

I am, one might say, a bit of an RSS junkie. It has been the main source of

news and information for me for about the last year or so, but I began to

feel as if I was drowning in it. I subscribe to something over 80 feeds,

some of which, like techcrunch, spit out over 40 or 50 items a day. A few

days away from my feed reader and there could be well over 1000 entries

waiting when I got back. I had them split down into lots of different

folders, based on interest, but it was just all getting too much, so I've

adopted a simple strategy which has totally gotten over the problem. I could

have just dropped all

I now only have 3 folders. Work Important, Personal Imortant and everything

else. I'll read everything in the first two folders, but the threshold for

getting into the first is very high, and the threshold for getting into the

second is basically personal blogs from close friends. Everything else goes

into the 3rd folder and if something doesn't jump out at me when I scan

quickly through this folder then I just delete everything. Now the time to

process all of these feeds is about 10 minutes.

I do have ideas about how to viksualise this river of information, but

perhaps I'll leave those ideas until I get a Google App account.

Read and post comments

|

Send to a friend

Efficient Name Disambiguation for Large-Scale Databases

2008-12-21T00:00:00-08:00

I've been interested for a while now in entity disambiguation,

particularly where the e

ntities are the names of authors in academic journals.

Jian Huang, Seyda Ertekin and C Lee Giles have a paper from 2006 in

which they describe

the method that they used to diambiguate the CiteSeer data set of

over 700,000 article

s. The paper is "Efficient Name Disambiguation for Large-Scale Databases".

They were able to use this algorithim to disambiguate this data set

over three days int

o just under half a million unique authors (though I didn't see a

mention of the hardware nor of whether they used a linear or parallel

computing approach).

Their approach seems to be to create an online SVM to bootstap a

distance funtion. This

distance funtion can be trained using a number of types of

information, names, meta da

ta such as emails, and terms extracted from the associated papers.

They then block author names into groups based on name similarity, use

the distance fun

vtion found with the SVM and find groups of names associated with the

same person by sc

anning over the data using DBSCAN, which is a clustering algorithm

that creates cluster

s based on a minimal distance and minimal number of members. By

slicing up the paramate

r space based on minimal distance, rather than on an a-priori number

of clusters, the a

lgorithim is insensitive to a change in the number of points in the

parameter space. Th

is means you can use this method in an iterative way and it can be

adopted to new data

as it arrives. I'm remined of some papers in astrophysics that did

clustering based on

voroni volumes, but only in so far as the voroni method is vaguley

related to a density

method.

All in all this looks like a nice approach to the problem, and the

authors got 90% accu

racy with their method, which is probably enough to bootstap a

solution into existence.

Read and post comments |
Send to a friend

Creationism as Science,

2008-12-21T00:00:00-08:00

I'm at the netsci08 conference and there is a really delightful talk

about the network of papers published in the creationisim/evolution

debate.

The group looked at key people in the ID debate and people who acted as

strong defenders of evolution. One can then make a graph of the links

between the groups and intra-groups based on citation and co-citation.

Now one area of social science that is pretty interesting in this debate

is looking at triads, as there are clearly going to be antagonitic

relationships in this debate.

In the graph evolutionists are blue, creationists are yellow and Dawkins

is red on his own.

From the graph it is clear that there are some people who are opinion

leaders.

Ok, what is interesting is that there are far more mixed triples than

similar triples in the graph, meaning that people from both sides seem

to be spending more time slagging each other off than agreeing with

their friends.

This is the first time that I would be tempted to say that the study of

creationisim could be considered science.

Read and post comments |
Send to a friend

Cloverfield

2008-12-21T00:00:00-08:00

Cloverfield

Originally uploaded by Ian Mulvany

My Work Colleague make a cloverfield diarama in our sandbox today. We will be going into full scale production soon.

Read and post comments |
Send to a friend

A minor milestone in leisurely reading

2008-12-21T00:00:00-08:00

Last night I finally finished Neil Stephenson's Quicksilver. Oh, don't

get me wrong, I've read it, and the full trilogy, before, but for much

of the past year I have been re-reading it aloud to my girlfriend. She

has an eye strain which means that if she reads too much at night she

gets headaches, and so I've been doing the recreational reading for

the two of us over the past year. We started out with Quicksilver, and

made steady but slow progress through it. There have been some breaks,

a small diversion through the worlds of Philip Pullman, a frantic dash

through the last Harry Potter, but quicksilver has been the constant

through the journey, a stable rock against which we could navigate.

There have been patches which have been hard to read out loud, many

sections of the book include deep philosophical discourse between

multiple characters, and when reading making clear who was saying what

was at times a bit of a challenge, but we finally got to the last page

just before midnight last night. Now just The Confusion, The System of

the World and The Cryptonomicon left to get through!

Read and post comments

|

Send to a friend

weekend grill

2008-07-14T00:00:00-07:00

I visited my girlfriend in Germany this weekend, and her father wascelebrating his 60th birthday. A friend of his arranged a full wild boarfrom the German forests. The spit it and slow cooked it for 7 hours, ittasted amazing. I can quite see why Asterix and friends got so excited aboutthis.

13102007410.jpg

13102007409.jpg

spreading of meme's behavior driven transmission of an idea

2008-07-14T00:00:00-07:00

Thistech blog article is a charming description of how human behaviour seemsto have spread the name of a particular wireless node across america. Ithink it is a lovely example of the spread of an idea driven by simpleself-interest.

sometimes, just sometimes a piece of news makes me smile

2008-07-14T00:00:00-07:00

From the Guardian today, a group of ultra-right wing MEP's were unable toform a group within the European Parliament owing to internal xenophobia.

http://www.guardian.co.uk/farright/story/0,,2211167,00.html?gusrc=rss&feed=12

restaurant BAYOU Berlin

2008-07-14T00:00:00-07:00

I got the email below a few days ago:

Hello to you all,Please forgive the bulk email! We hope all is well with you.We will be opening our new restaurant on December 1st. Please visit our newwebsite at www.bayou-berlin.com You are able to book hotel reservations onour site using our corporate discount rate. We hope to see you sometimesoon,All the best,Steve and Brandon

Earlier this year I was on holiday in Nerja in the south of Spain. On one ofthe last nights of the holiday we went to an amazing Cajun restaurant calledCafe New Orleans. I got chatting to the two owners and they told us abouthow they had been trying to sell their place in Spain and open up inGermany. Well the food was great, and the guy's running the place were superfriendly, so I was delighted to get this message as it's clear that theyhave finally made their place in Germany happen. If you find yourself inBerlin then you should definitely consider giving this place a try.

pythonic is a way of thinking

2008-07-14T00:00:00-07:00

Thistutorial delves into pythonic ways of coding. I'm learning from it.

tags: python, hoto, tutorial

open science

2008-07-14T00:00:00-07:00

I just posted the below as a comment on a blog, but it was good so I thoughtI'd repost here

What is open science and what is the system? Well I am sure that there aremany viewpoints on this, so I am going to just put forward one here.

At a fundamental level 'the system' is how we ascribe credit toparticipation in science. The credit is converted to grant money, thedollars keep the food on the plate. The decision makers for the grantsgenerally lie at national funding level. These people are busy and have alot of applications to process, but that is not to say that they aredisinterested in the state of the scientific funding ecosystem. However aslong as there are too many decisions and not enough time then metrics suchas a measure based on journal related factors will dominate. If it is easyto measure and to see how credit can be assigned for contributions that lieoutside the traditional publication at the end of the research cycle then Iam confident that such criteria will be taken into account, but it is veryearly days at the moment and I don't think we can expect to see an overnightrapid transition, especially when the tools for measuring such contributionsare in their infancy.

What is open science, and why might it be important to funding agencies tosee it being utilized? Again, just one viewpoint. Well, there is a lot ofdata out there, and I expect that a lot of good science could be done onsecond hand data. This is already common in Astronomy. This should help toutilize efficiencies of scale. In addition with better information aboutwhat is happening, and more eyeballs working together, the amount ofredundant work can hopefully be minimized. As the open source adage goes,with many eyeballs all bugs are shallow.

There are also a lot of published papers out there, and the scaling time foran individual to get to the data resource that they need is only going toget longer when there is more information to process. I recall hearing thatin 2005 something along the lines of 600,000 people graduated in China witha degree in engineering. If you talk to any academic journal editor most ofthem will tell you that the rate of submission of papers from the pool ofChinese scientists is growing year on year.

I see a function of open science as being a way to help the flow ofinformation in an open system that maximizes the efficiency for the rightpiece of information to get to the right person, whether that be a piece ofdata for analysis, a protocol for an experiment or a contact for acollaboration.

We have a prerogative to make this happen as a consequence of excess ofinformation that we are faced with.

There is however the very important need to be able to credit people whoparticipate in an open way. As someone working for an academic publisher Ifeel that part of my job to help create systems that can help to moreaccurately measure broader contributions to the scientific enterprise. As Isaid earlier, these systems are in their infancy, but it is a very excitingtime to be involved with this.

tags: science, science2.0, open science

mozy backup service

2008-07-14T00:00:00-07:00

I've been trying the free account this week. I've backed up 1.6 GB of data.It took, with given interruptions of a normal work day, about 4 days tobackup this amount of Data.

I am beginning to think that the world of offline remote backups is still alittle bit away!

live blogging from BarCamp Cambridge, Matt's talk

2008-07-14T00:00:00-07:00

Matt talking about semanitic web for science, an introductionXML, URI, namespaces, RDL OWL,standards are often argued about but it;s just XML

we are supposed to be able to publish semantis data easily,at the moment it's not just an extension but a whole other world,people won't learn sapqrl

Matt believes that we can get the benifits of semantic now, but withoutin any case, it's hard to get funding

should consider semantic web, rather than Semantic Web,

how do we add sematic value to existing dcuments,enemble is the public interface originally to hte human genome project, butthere are lots of other gnese in there nowset up to fight against the patenting of genescontains microformat in web output now!enembleit's open source and open data

people understand how to look at a html source, so it's easy to add, thevalue is there without the overheadjust add standard html classes, and let people do what they want to do withit.Q - will this lead to islands of parsing?

The idea is enseble is a big resource, and hope that people will follow socreate a defacto-standard

Can style it,Parse it

can the website be the API?

can use a standard uri to access the data

cut down on the amount of code that gets written

there is more data on the web, then is available through the api(we are not the only ones)

Q - What if your api is just your searchFlickr and Yahoo do this

Why not a pipes fo biology?

see microformats.org

have started abioformats.org

the microformats approach is slow, they have the idea of the process,then you have to go through a standard, and it goes round after round,we should just get started(see operator plugin, the browser becomes the broker for data)

slideshare.net/mza

live blogging from BarCamp Cambridge, Matt's discussion

2008-07-14T00:00:00-07:00

discussionQ Gridle is raised along with RDFa

if you don't like the way that the xslt is workin you can make your own.however most domain exprts can't write xslt

if you just let the domain experts create microformats you may leave theontologicaldefinitions to people who are creating the xslt

AG says that there may be two different problems, address to addressbookfrom page vs data harvesting

live blogging from BarCamp Cambridge, Laura James

2008-07-14T00:00:00-07:00

Laura JamesAlert Me.Com

might get too corporate.

trying to do the internet of things, internet access to small devicesthey are implimetning today, and will be shipping later this year.comes from R&D, but working in a shipping

they are going to ship a home security system, but they are actuallybuilding a platofrmthat can connect anything that does not require full audio and video

using a mesh network that connect to a hub using a 'zigby'output can be things like a lamp that has a color dependant state

the hub runs on linux with python on toprun by xml doc

can do things like tell you what day is bin day

is mains powered with battery backup

plugs in to ethernet

sounds just too cool

connects to a hubserver

they have gprs to connect to the internet if the router goes outtalks to a dialog server and a DB

there is data from loads of sensors in the homegoes to the hub,the hub has a logic enginego to the website and set it up to let you know

can send you a twitter when your doorbell rings

there are two really big trade offssecurity vs usabilityneed to make sure that your home can not get hackedand that your data does not get leaked

e.g. don't want people to have to type in the mac address of every entity inthe network

reliability vs extensabililyt

networ at home is based on zigby, low powered wireless networkopen standards

smallet item is a zigby tile, about 2cm square. range will be about 300mwith one tile.ca n cover a standard home with ome base station.

Q- could you set this up for a wet lab or other lab?

live blogging from BarCamp Cambridge,

2008-07-14T00:00:00-07:00

I am at Bar Camp Cambridge,

We have had the three word introductions and are just running through themorning talks now. It's pretty cool.

The coffee is good, and cookies are great.

Let's see how the day goes.

google reader now has search

2008-07-14T00:00:00-07:00

Well, OK, this is probably not news, but I guess what is interesting for meis that this is a feature that I didn't really notice until today when Iwent looking for it, and hey, though that little search box had probablybeen there for weeks now, I only "saw" it today when I looked for it.

getting started

2008-07-14T00:00:00-07:00

Well, i'm starting out my mob-blogging life waiting in que for a flight to milan, and i discover that my predictive text thinks i'm a clogger! An interesting, if somewhat disturbing begining. Tags: moblog, txt, travel

get addicted, feed the world

2008-07-14T00:00:00-07:00

at http://www.freerice.org

friends of mine of TV

2008-07-14T00:00:00-07:00

A friend of mine set up her own business a few years ago, at the same timeas having a baby. It was really tough for Ais for the first couple of years,but it is looking like things are really working out for her now. She wasrecently featured on TV in Ireland, and she has posted a link to the clipfrom her blog. For a really inspirational view on managing business withchildcare have a loot at the vid:

this link:http://chewingpaper.typepad.com/chewing_paper_confessions/2007/10/tv-fame.html

tags: business, cool, inspirational

comment on climbing goals

2008-07-14T00:00:00-07:00

I find that having a cool trip planned every now and again is key forkeeping me motivated. I've been doing a lot of job and country hopping overthe past 10 years, and this has had a signifigant detrimental impact on theamount of climbing that I could do, but hey, I'm doing a job that I love,and those are the rolls we choose to take.

One thing that has kept me motivated to get of my arse and actually try abit harder when I am down in the gym is having a few cool trips planned inthe year. I'm not a major road-trip warrior, so I can only usually manage afew long weekends a year, and maybe one longer trip, but the motivation itgives is powerful stuff. I'm about to head to font for a week att hebeginning of September and that has me well primed right now.

These trips generally are my short-term goals, and I keep track of mypersonal bests. I know which problems I sent last time I was in font, andmore importantly which ones I nearly sent (oh yes, they will be mine!!).

Long term goals I must admit to having a problem with. My mid term goal forclimbing is to try to stay in one country for a few years and to get adriving license. Funny, but it seems that this is just the way a lot of mygood friends got better.

and while we are at it here are the references from that article

2008-07-14T00:00:00-07:00

References

- "Python Objects", Fredrik Lundh, http://www.effbot.org/zone/python-objects.htm - "How to think like a Pythonista", Mark Hammond, http://python.net/crew/mwh/hacks/objectthink.html - "Python main() functions", Guido van Rossum, http://www.artima.com/weblogs/viewpost.jsp?thread=4829 - "Python Idioms and Efficiency", http://jaynes.colorado.edu/PythonIdioms.html - "Python track: python idioms", http://www.cs.caltech.edu/courses/cs11/material/python/misc/python_idioms.html - "Be Pythonic", Shalabh Chaturvedi, http://shalabh.infogami.com/Be_Pythonic2 - "Python Is Not Java", Phillip J. Eby, http://dirtsimple.org/2004/12/python-is-not-java.html - "What is Pythonic?", Martijn Faassen, http://faassen.n--tree.net/blog/view/weblog/2005/08/06/0 - "Sorting Mini-HOWTO", Andrew Dalke, http://wiki.python.org/moin/HowTo/Sorting - "Python Idioms", http://www.gungfu.de/facts/wiki/Main/PythonIdioms - "Python FAQs", http://www.python.org/doc/faq/

agile programming

2008-07-14T00:00:00-07:00

I work for a department where we try to do agile development. I liked thisDilbertabout the subject.

Weddings make you fat

2008-07-14T00:00:00-07:00

I just got back from a wedding and bier festival thing in Munich. I've been tracking my weight on and off for the past year and a half with the ultimate goal of getting back to my climbing fit weight of about 80kgs. From the graph below you can see the spike that was caused by the extertions at the wedding!

(OK, in the vox rendering you can't really see the spike on the right hand side of the graph, but if you are bothered then click through and you'll see the original graph. btw the y-axis is a measure of how much weight I want to lose)

Today at lunch duplicators and instantiators

2008-07-14T00:00:00-07:00

I work at a pretty innovative place and the discussions that we have atlunch time are pretty mind bending. The last time I can recall having somany interesting discussions over lunch and breaks was way back when I wasin my first undergraduate degree, but the topics of conversation that tendto come up at my current place of employment are really worth trying torecord.

Today we started off having a couple of split conversation, Euan describingchatting to Martha at Sci-Foo and Euan and I talking about Euler, as you do.Then the conversation quite naturally turned to replicators. We werediscussing the pro's and con's of owning a replicator. It's clear from theconversation that two calsses of replicator exist. One would break the lawsof physics and replicate anything out of nothing (possibly dragging theextra matter out of an alterantive universe, vis a vis the lazy gun inagainst a dark background) and the other type the just melds matter into theshape of whatever might be copies.

Brenda wanted one that would not copy living things, but as someone pointedout, if you could copy living things, and you had one kitten, then you couldhave kittens!

Getting a transfer of ownership form when you copied a beamer might be aproblem, and if you copied the earth then you would be in right trouble.

At the end Gavin wanted one that you could just pass science fiction booksto, and it would make stuff from the book not really a replicator then, butan instantiator.

tags: lunch, replicator

Tips for switching from Windows to Mac

2008-07-14T00:00:00-07:00

I've written up the short answers to some questions that you might have if this is the first time to use a mac, read through this and if you followthe suggestions you should have the hang to the system pretty quickly.

If it turns out you don't like it, then at least you will have made an informeddecision!

-- Where is the right mouse button?

This is the biggest obvious first difference. There are four buttons on the bottom left of the keyboard that I am going to call 'fn', 'ctr', 'option' and 'mac' respctivley. If you need to press two at the same time then I am going to write it like "ctr + fn", for example.

The right button mouse action equvalent is hoilding down "ctr + mouse button".To get the mouse to do something you generally have to double click on the thing you are clicking on. One click is a selection, a double click is an action.

-- I double click on an application but I want to find the preferences, or othermenus, and they are not on the application window, how do I do that?

The very top bar of the mac changes with applicaiton. The menu bars are all along there. You will see a small blue apple in the very top left corner. The name of the application that is active is frst, and if you clioc on that you will get the option of the prefereces panel for the application. You canalso try "mac + ," (yes, that's the mac button and the comma button together,it tends to work for all mac applications, joy!).

The top right of the bar gives you information about how your system is doing, battery, internet connection, time etc. You can load this baby with all sorts of things.

-- Where are all of my files, where are the application?

The OS is based directly on a UNIX system, and it's file structure is the same.If you double click on the Macintosh HD in the top right you will see a filebrowser with some directories. This opens a the very botton level of the file structure of the machine. ALL of the applications on the macine are in the Applications folder (actually, there are a host of very loe level system appsthat are not in here, they are in System, best not to go in there!).This machine can host muliple accounts, and these are in the Users directory, double click on this now. You will see a list of account names, yours "John"has a little picture of a home, this is your home directory. Try clicking in the other accounts. Some folders are locked, others you can have a look around, but you can only create files and delete files from your own local folders. Let's have a look at those now. Amongst the default folders in youraccount the most important is Library. This stores all of your system preferecesfor all of your applications. That means two people on the same machine can usethe same app, but have totally different setups. Your girlfriend on her accountwill never see the porn sites that you were navigating too in the webbrowser, forexample! For the moment it is probably best to stay out of there.

-- What are all of those things on the botton?

That's the dock. It keeps a ttrack of open stuff, I don't use it much, but it can be good if you want to keep a few applications to hand that you use a lotrather than going into the applications folder each time. I'll show you how to customise that later.

-- Some useful quick key commands and window browsing tips.

"mac + s" = Save"mac + c" = copy"mac + x" = cutpaste is "opt + p + ,". No, kiddind, its "mac + v" of course! (the mac key predates that windows key on PC's).

Now I want you to open several windows of a document. Double click on any applications. Do it a few times. Make sure to open Safari (it looks like a compass) and when you have hit "mac + n" a few times. Now you have lotsof applications open on your desktop. Clicking on the orange button will minimse the window and put it in the Dock. Clicking on the mini-windowin the dock will make it pop up. Cool, eh! OK, Now do that again, but keepthe shift key pressed at the smae time. It does it slowly, just so you can show off!. Now try the following key presses:

"mac + `" - cylces through open windows of a given applicaiton"mac + tab" - lets you pick between applications"mac + shift + tab" - same as above, but in the other direction"mac + 1" - unminises. If you have 3 windows of the same app minimsed, then you can do "mac + 3" or "mac + 2" to get the specific window that you want."mac + h" - hides an app. All of the windows of the app dissapear. They don't go to the dock. To get them back either click on the app icon in the dock, orcycle to the app using "mac + tab"."alt + mac + h" hides all windows apart from the one you are working on.

Ok, now for some really cool stuff. "fn + 11" - while you keep these two pressed all your windows head out to the edge of the screen so you can see your desktop."fn + 10" - exposes all of the windows of a particular app. click on the window you want makes it come to the front."fn + 9" - exposes all windows. "fn + 12" - opens up the dashboard. This is a applish thing with lots of widgets to choose from.

"mac + q" will quit an application. Using "mac + tab" and keeping mac pressedyou can quickly kill all of the applications that you have running."mac + delete" will delete a file"mac + w" will close the current window you have open.

-- How do I find something?

"mac + space" - opens the spotlight window. It is really good. Really really good. It means you really don't have to look for anything with pain any more.I have 250 GB of data indexed by spotlight, and it finds things instantly.

-- No, I'm sorry, this really isn't cool enough, show me something more.

Ok, Mac now has smart folders. I have included two on your desktop. You define thekind of file that should go in there. As soon as a file anywhere on your computer that matches this criteria gets created it gets listed in the smart folder.

-- kinds of free applications that you can get.

There are loads. My favourite is called Quicksilver. I have set it up torun for you on your account. Another is called Desctop Manager and I have also set that up for you.

The Desktop Manager provides as many virtual desktops as you wnat, I haveset it up for 4 for you here. A virtual destop is just some extra realestate.The four small squares in the bar on top of the screen show you your desktops in minature. The small squre bottom left of the screen is a pager, and it does the same job as the squares on top. I normally only use one or the other, you can set up how you want it in the preferences. Clickingon a different square to the one that you are in will bring you to that desktop.You can also use the key command "alt + mac + ->" (right arrow) or"alt + mac + <-" to cycle through the desktops.

Type "mac + enter". This is quicksilver. Actually, It's not working right now, but i can show it to you on my account where it works very well at the moment.

-- Hmm, someone sent me a Microsoft file, how do I open it on this totally bitchin machine,

Chances are that you already can. Try double clicking on it to see what happens.The application NeoOfficeJ does all of the same things that Microsift Officedoes, oh, but it's free. I have set it up on the machine and on the dock, it's the one with the sailing ship icon (?)

-- Gah, something really has gone wrong, and I need to force quit an application.

Go to the apple icon top left. One of the menu options is Force Quit, there you go.

-- Oprn iTunes, I dropped some cool tunes from my collection there for you.You can try burning them to a CD if you have one, it's easy, but I'll let you figure out how.

The fruitcakes have a talking form

2008-07-14T00:00:00-07:00

A new creationisim journal ARJ.GRRRRRRRR

The Laboratory Website and Video Awards

2008-07-14T00:00:00-07:00

Attila Csordas sent along links about these awards, looks nice,

http://www.the-scientist.com/lawva/

and a blog link about them:

http://pimm.wordpress.com/2007/10/09/the-laboratory-website-and-video-awards-by-the-scientist

tags: science

Simpsian

2008-07-14T00:00:00-07:00

Yesterday one of the guy's at the office interviewed the producer of TheSimpsons. As a result he had to do quite a lot of research yesterday onSimsonalia, and decided to use the make your own simpson avatar tool on theSimpsons movie page to make all of us our own simpson character, this iswhat I would look like if I were a simpson.

Tags: simpsons, avatar

IansSimpson.jpg

Sci - foo lurking

2008-07-14T00:00:00-07:00

I'm a Sci-Foo lurker, as I didn't make the cut to go this year, but I'vebeen lapping up the news seeping out of the meeting. So far my favoriteTwitter post about the meeting comes from Richard Ackerman's feed:

"Timo and Tim are trying to kill us. Doesn't cult brainwashing start withsleep deprivation? Fun so far, anyway"

Potter prediction

2008-07-14T00:00:00-07:00

I wonder how long it's going to be before we see a children's TV series setat Hogwarts. I predict within the next 5 years.

tags: Harry Potter, TV

Patents and Peer Review, kissing cousins?

2008-07-14T00:00:00-07:00

Below follows a long comment that I posted to an article on O'Reilly Radarabout patents.

The original blog post is here:http://radar.oreilly.com/archives/2007/09/three_vantage_p.html

There is an interesting compliment to the patent system in the domain ofassigning credit to ideas, which is the academic peer-review system.

The inventor of the idea in the case of academia is the author. Theequivalent to the patent office is the editorial board of the academicjournal that the author submits to.

There is an idiosyncratic historical connection between the two systems too.Einstein famously worked as a patent clerk in Bern, Switzerland, beforebecoming a published academic.

At the time that Einstein was working the volume of patent applications wasprobably on a par with the submission rate of academic papers to peerreviewed journals, though I have no figures to back this up. In both casesan idea was submitted, examined by experts for originality and eitheraccepted, granting the creator rights, or rejected, sending them back to thedrawing board.

Fast forward to today and it seems that on the whole the two systems, workin almost the completely opposite manner from one another.

The peer review system continues to work much as it used to. An idea issubmitted, reviewed, in most cases rejected initially, tweaked, re-submittedand possibly accepted. Scientists for the most part are looking for peerrecognition though publication, and the career rewards that come with thatthrough tenure and increased likelihood of earning academic grants. Themicro-bio and nano fields are diverging from this template, but I'll getback to that at the end. I would classify this system as one in which theinterrogation of the idea happens before the laurel is bestowed on theapplicant. Contrary claims or claims of prior art appear as new publicationsin an ongoing conversation, but it is very rare for the reward (the citablepaper) to be retracted. Working as a publishing editor for three yearsmanaging five academic journals I saw this happen with one paper, and eventhat that was considered unusual.

The patent system currently seems to work in completely the opposite way.The bar for having a patent approved seems quite low, but even afteracceptance of a patent the rewards are not guaranteed until after the patenthas been challenged in a court. The interrogation happens after the laurelhas been bestowed, and only in cases where there is contention that theremight actually be money to be made, which seems to be the real reward thatdrives people to patent their ideas rather than publish them.

Now why is this the case? I would offer this idea. Patent offices areessentially national bodies. This means that compared to the growth ofacademic journals there are relatively few of them. The academic systemprovides a distributed means of testing the goodness of the ideas, and so asthe volume of academic submissions has risen the journal system has beenable to scale with the growth and been able to cope.

While patent applications remained mainly technical it was possible for acentralized office to cope with the volume, but the advent of softwarepatents has dramatically changed the balance. (I'm not saying that theseideas are essentially less valuable intrinsically, but rather that theexplosion of people literate in creating though the meduium of computer codehas increased dramitaclly the volume of patents that are being filed). Underthis increase in volume the national patent office system has simply broken.Where it now finds it's scalability is in the court system. There are lotsof courts and lots of lawyers versed in patent law. I am not sure that anyfix to the patent office, or to the information that is required to submit apatent is going to get rid of the problem of scalability as long as theoffice remains a centralized organization, but I might well be dead wrongabout this.

In what could be a worrying parallel to the way that the patent system hasgone the peer review system is beginning to experience strain. It is clearthat it is key in the scientific process, however the vast numbers ofqualified scientists coming out of China and India are beginning to greatlyincrease the submission rate to academic journals. The peer review system isbeginning to creak under the weight. I hasten to add that it is working justfine at the moment, but consequences can be seen in the growing importanceof various citation indiceies. A publication on its own is no longersufficient for the advancement of most academic carerrs. It usually has tocome with additional properties, such as being in a popular journal, beingcited a certain number of times.

Another interesting development is that in some scientific areas patents arereplacing peer-reviewed publications. I know of a few groups working onnanotechnology where they have results that could be published in a peerreviewed journal, but to do so might infringe on IP. In other cases therevenue from industry that the groups are attracting mean that at the momentthey don't have the time to commit to writing up their results forpeer-review.

Are the two systems getting closer again?

Could the patent system benefit from having different levels of quality inthe way that the peer-review system has different quality journals? Agold-standard patent requiring full code disclosure and bug-free runningprogram, with a bronze-standard patent being equivalent to the currentlevel?

How can both systems utilize collective intelligence to alleviate thenumerical and informational pressure that surrounds the act of review?

Though a word of caution, if you have answerers to the above questions youshould probably patent them, shhh now.

Open Reviewing

2008-07-14T00:00:00-07:00

Arecent blog post on Action Potential pointed me towards the neuroscience peer review consortium. Theyhave a description here about theconsortium and a list ofparticipating journals. I have no doubt that this is the future of peerreview. At the moment the peer review system is horribly inefficient forpapers that get rejected. Rejection from a journal can occur because a paperis crap, but often it happens for many other reasons, because the journalhas already met it's pagequota, the journal is publishing a set of special issues on another topic,the editors of the journal are interested in shifting the focus of thejournal, the topic of the paper is slightly away from the main interests ofthe editorial board, the paper is good, but just gets edged out by a set ofbetter papers that come in. The author resubmits, rewrites, reformats thepaper for another journal, the review process happens again, sometimes withthe same reviewers being approached again to re-review the paper. Thismultiple reviewing of the same paper is for the most part a waste of time.There are of course cases where it is good to have the option, but for themost part it wastes the time of communities who are highly trained and busytrying to do important and original work of their own.

I personally always felt that pool reviewing was the natural solution tothis problem, but I never thought I would see it happen owing to thecompeting nature of academic publishers. This attempt to work pool reviewingin one area is very exciting. The issues that need to be overcome includethe issue of private reviewer comments that the Action Potential blog pointsout, but also the questions of a set of editors of one journal being willingto take the rejected papers from another journal. One would like to say thatin a totally equally fair system all journals are equally important to thedevelopment of science, but that is not true. Some journals are just moreimportant for one reason or another. When two journals feel they are betterthan each other are they going to be willing to take the papers rejected bytheir competitors? I hope that the papers will stand up for themselves andthat this system will lead to a more efficient matching of papers to pagesin published journals.

If it works what next? One single format for submission? One passpeer-review with a peer-reviewed preprint publication guaranteed for allpapers? I don't know, but I'm interested in seeing what happens with this.

tags: science, peer-review, science2.0

New Terminal In Mac Os 10.5 Is Nice But Dont Forget That There Is A Bash_profile Out Of The Box

2008-07-14T00:00:00-07:00

---

layout: post

title: New terminal in Mac OS 10.5 is nice, but don't forget that there is a .bash_profile out of the box. categories:

- mac

---

The new terminal is very nice, but I did a wipe and re-install and waswondering what had happened to my old terminal settings. Of course thedefault .bash_profile file was being used as the settings file, so I deletedit and now the terminal works as expected.

******************************************************************************** DISCLAIMER: This e-mail is confidential and should not be used by anyone who isnot the original intended recipient. If you have received this e-mail in errorplease inform the sender and delete it from your mailbox or any other storagemechanism. Neither Macmillan Publishers Limited nor any of its agents acceptliability for any statements made which are clearly the sender's own and notexpressly made on behalf of Macmillan Publishers Limited or one of its agents.Please note that neither Macmillan Publishers Limited nor any of its agentsaccept any responsibility for viruses that may be contained in this e-mail orits attachments and it is your responsibility to scan the e-mail and attachments (if any). No contracts may be concluded on behalf of Macmillan Publishers Limited or its agents by means of e-mail communication. Macmillan Publishers Limited Registered in England and Wales with registered number 785998 Registered Office Brunel Road, Houndmills, Basingstoke RG21 6XS ********************************************************************************

Money Saving Web Sites for Living in the UK

2008-07-14T00:00:00-07:00

The Guardian ran an article this week listing 10. As a placeholder here aresome of them that I might find useful:

https://www.mysupermarket.co.uk/

http://www.saynoto0870.com/

http://www.quidco.com/

http://www.energysavingtrust.org.uk/

Mining social data, what Flickr tells us about place

2008-07-14T00:00:00-07:00

toscany activity spikes with R Originally uploaded by fgirardin

Web 2.0 is a buz word. At a conference that I attended last week it came up frequently as being one of the most over-hyped terms around at the moment, but there is substance behind it, and how that substance appears is fully up to the creativity of people who are willing to get their hands dirty with the actual data that is being produced by social web sites. I came across this example today and it exemplifies the type of thing that can be done.

These researchers are looking at time traces of pictures of locations uploaded to flickr and pulling information about those places out of this data. The image here shows activity in and around tuscany. They are also looking at routing through contries and heat-maps of tourist visits to cities.

You can read more here:

http://www.girardin.org/fabien/tracing/

Liam dropping by london

2008-07-14T00:00:00-07:00

10072007136

Installing Python Image Library on a Mac

2008-07-14T00:00:00-07:00

I spent about half an hour trying to install the excellent Python ImageLibrary on my mac. It's one of those things that is more annying than itshould be. You get libjpeg from herehttp://www.ijg.org/files/jpegsrc.v6b.tar.gz follow the instructions here onhow to patch the makefile and how to install libjpeg:http://www.kyngchaos.com/macosx/install/libjpeg (all this after installingthe developer tools on mac), only to fail and then discover that there is aprebuilt binary package here:http://pythonmac.org/packages/py25-fat/index.html which works perfectly. GoLena!

If programming is magic, then the old languages are strong

2008-07-14T00:00:00-07:00

The similarity between magic and programming is never more apparent thanwhen you sit down and watch someone who is truly good on the command line.By virtue of having secret knowledge they can quickly accomplish theapparently impossible. If you start out in the unix world you begin bylearning some simple incantations such as 'cd', 'ls' and 'top' and 'kill'.If you persevere with the art eventually you may master the hidden runesthat bring 'sed' and 'awk' to life.

These sorts of commands work at the very root of the operating system, hardwired into the c++ code that built up the universe in which you are walking.In a way this is the fundamental elemental magic of programming. And what Ifind so pleasant about wandering in this landscape is that findingdocumentation on these building blocks is so easy. They have been around forsuch a long time that much of the documentation was written when blue links,plain text and white backgrounds were not just a style issue for html, therewas simply no better way to do it back then.

I often think that there is still no better way to do it.

tags: programming, magic

Great places to hang out in Den Haag

2008-07-14T00:00:00-07:00

I have a good friend who lives in Den Haag and passed on the followingadvice about places to go and eat if you have a small child. This is reallygreat advice so I thought I should post it here:

Hi Victoria,

Very sad to hear your friend has such a lousy experience in my country. AsIan can confirm, it's not a very easy place for 'outsiders'... The upside isthat if you are not at fancy free and footloose as my friend Ian is- andwith a little boy that age she will have moderated her dancing ambitions-The Hague can be quite a nice place to hang out.

Here are a couple of my personal favorites:

- Ziani: italian restaurant espically great with children. There is even alittle playcorner and no one will mind a couple of screeming children (Ihave experience):http://iens.nl/restaurantsVan/DenHaag/restaurant.htms?r=4240

- De Boterwaag: grand cafe at the center of town (Grote Marktstraat). Goodplace to have a few beers and especially sunday afternoon it is crowded withchildren. Bit smokey but all that will be history after jan 1 when allpublic hangouts will be smoke free. Remnds me that I should quit beforethen.

- Bodega 'De Pakschuit'. On the canal on the old part of the center. Goodtapas and again fine place for drinks on sunday afternoon with children. Nottoo crowded.

Plenty of good parks and playgrounds of course. I frenquent the playgroundon the Scheveningseweg/Kerkhoff laan a lot. Good stuff for kids and daddycan look at the skateboarders when he is bored. She should definately checkout the park 'Clingendael', very classy park and good playground as well. Iam sure she will have discoevered the marvels of the Beach, and it isstarting to become off-season but I highly recomemend strandpavilioen 'Zuid'on the 'Zuiderstrand'.

Best thing of coursee with children is to get a babysit and have a romanticevening out for mom and dad.

- Pastis: french bistro at the hart of the old center. Decent steakbearnaise and good ambiance:http://iens.nl/restaurantsVan/DenHaag/restaurant.htms?r=21430

- Favorite Thais restaurant is Ponsawan:http://iens.nl/restaurantsVan/DenHaag/restaurant.htms?r=18512 great food andvery frendly people (right around the corner from where Ian used to live butlet's admit it Ian, you were also a bit shortsighted).

For mom alone, when she is tired of the whole afair: Wicked Wineshttp://iens.nl/restaurantsVan/DenHaag/restaurant.htms?r=5571

Nice place that women tend to like on wednesday nights for a glas and somegossip.

For Dad, when he just wants to get hammered on Belgian beertasting: Cafe 'DePaas' on the Dunne Bierkade. Ask Ian (he still will recall that he beat meby one glass and was very proud of that. Never saw him check in the officethe next day though).

Getting motivated for climbing

2008-07-14T00:00:00-07:00

A good friend of mine has just hit the 8a mark in climbing. Dave is a greatclimber, and a really nice guy, but what really blew me away is that hisfiancee Caroline led 7b on her first sports climbing trip.

One thing know about both of them is that they do a lot of running,something like 10 - 15 miles evrey morning.

If this isn't motivation for me to pick up running then I don't know whatis.

tags: training, climbing

Get Slimmer Pets

2008-07-14T00:00:00-07:00

12/01/2008 Originally uploaded by Ian Mulvany

I was shopping at Lidl last weekend and saw this for sale. It might be a bit of a drastic measure, but I think it would work.

CPANPLUS and Mac OSX

2008-07-14T00:00:00-07:00

OK, so I'm interested in installing the Connotea code base on my mac fordevelopment reasons. It will be the first time that I am going to startplaying around with a largish perl installation on my macine and there are alot of perl dependencies to be installed. Rather than going through CPANmodule by module, and rather than writing a bundle file for the connoteadependancies (because I don't know how to do that yet) I decided to try touse CPANPLUS, due to the tips on perl administration withCPANPLUS

My first attempt to instal CPANPLUS didn't seem to get very far, then I readthat someone had succeeded by installing it into /Useres/Shared as root, sothat's what I tried and it seems to work. To be safe I ran the followingcommands on the advice of the program itself:

$ perl bin/cpanp-boxed -s selfupdate dependencies $ perl bin/cpanp-boxed -s selfupdate enabled_features

BarCamp Cambridge Jeff Fates, Drupal

2008-07-14T00:00:00-07:00

OS CMS systems beat the crap out of the free ones for what you get for your money, including support.

If you have a budget then you can get in touch with the authors of the OS systems easily

the only thing they sometimes don't win on is polish

it is someone's job to look at each piece and make sure that it is slick

Drupal is free, It upgrades about twice a year, one major one minor

is built on PHP and MySQLscales pretty well, but perhaps not as well as to the size that sanger would needbut does scale on small hardware to 100s of thousands of items and users

is very modular.

it has lot's of modulescore modules are very well written

It is very flexible. the core bit of content is a node,any content that you have, if it is a node, then it inherits a lot of featuressuch as getting commenting, revision control, access controlcategorisation,

CCK is the content creation kit

Views module is for making custom lists of pages and custom lists of notes

for example in hte drupal site there is a blog module, but you can make any custom view with the views module

overview of admin pageand module view, the view is quite hard to see as the light is a bit high in the back of the room at the moment.

There is some disagreement about the status of Casablanca as a great movie, suggestion gets derisory snort from Matt, ces't la vie.

a staging system would be nice, but is not there at the moment

BarCamp Cambridge - teacking computers to understand text, Peter Corbett

2008-07-14T00:00:00-07:00

a desk at the computer lab and at the chemistry lab.

computationl lingustic chemistryauto-detect language in chemistry papers to try to recognics chemical andmarkup.

suppliment the mark-up from publishers.

can draw the chemical and annotating them overlayed over the paper

some problems are that there can be new names in papers,comapct names, include extra hyphens, this program can deal with these kindsof things.

also can use systematics parsing.

this is the core technology, you can do things like search for alkloids inyour paper, or document dump

this seems to run within a browser.

run the software over a corpus of about 100 papers, and created a searchengine out of this?? I Might be wrong about that.

can create an svg

can go from plain text to something like a connection layout using aninformation rich markup

the RSC is using this software along with human-clanup to create markup ofchemistry papers.

can then to semantic search over papers.

Small natual languge processing trickimage we were interested in opiates,we could just ask opiates to googleyou can ask a question like "opiates such as" will give you a much betterreturn on results.

I just checkd this ad it works

there are many patterns like this, they are known as hurst patterns.

he did a pass over abstracts on pubmed for these kind of patterns to make anetwork of relationships

there is not a connected graph

dot failes on large graphs, but the demo does show that you can automate thediscovery of reaction networks.

you can do reasoning on structure as well as process (now he mentions lot'sof chemical names that I know nothing about)

a few bits of wisom from this

most of the informaion has come from biochemists rather than chemists,more biologists are into open science, and open databasechemisty has ben mostly captured by commercial interest,hard to get free chemistry data.

next is to define what you are looking for?you want to be able to evaluate how well the software has donehow do you post-annotate the documents?in a lot of text there is a diffeernce between what you think the worldlooks like andhow it is described in the literature, so even when you get people to ..

question about confidence levels,the most recent piece of the software has confidence levels. rare eventsdon't providegood confidence levels

it could depend on what you are looking for for,

Peter thinkgs that confidence is important for these systems

e.g. "a such has b" if b might be a chemical but you are not sure. if lateryou find in your search that a is indeed a chemical it raises yourconfidence that b is indeed a chemical

Q: is there any way to automate the acronyms of chemicals.turns out that this is not allways nice. you can do some of this.

BarCamp Cambridge - James talking about HTML5

2008-07-14T00:00:00-07:00

James is just an interested bysander on the HTML 5 mailing list, hey, it's abarcamp

html5 is th enew verison of html

applemozillaoperaanyone who joins the mailing list

and

w3c (which means MS, which means this is going to work in IE)

if you have ideas, then you can joing hte mailing list and put ideasforwards for the specification

why should we?

lots of information is locked up in HTML, not XML, not SVGL

most of it is invalid

it's important to know how to parse this invalid html, at the moment all ofthis understanding is locked up in browers, you have to reverse hack mozillaor IE souce, not a good situation to be in

HTML4 is underspecifiedincisistentdoes not match reality

for example the reason when google maps is launced, it didn't work in Safaribecause no one knows how to parse HTML

another example is video, you need a proprietary plug in to watch videos inyou tube, this is nuts

what of xhtml, this requires XML, and for a lot of people this is also nuts

most browser vendors can't impliemtnXHTML2 inther browser

what is the proceedure?identify use casesand look for solutions to use cases

this is more contentions than you would think

html 5 looks like html

some changes

doctype is shrtercharset is supported

what interesting features are implimented

things like

nav could be ignored by screen readers

aside is designed for pull out boxes.

some of the reasons for these new items is a google search for favouriteclass names used in htmlthese items closely follow a hughe number of entities that are already inuse

html 5 specifies more algorithms in more details

you can associate a visable caption with an image(is called a legend, not caption for historical reasons)

finally good support for video multiple encodings supported with source elements with fallback content

autoplay attribute for audio

lots of DOM support, so you could write a media player in HTML 5

this is working already

new inline elements, e.g. datetime, progress, meter. (specify value throughattributes or get values via program??)

lot's of support for forms,sample shows many high level form inputs with easy coding.

lots more, canvas element used by yahoo pipes

parsing, HTML privides a detailed parsing algorithim that can deal withmis-formed htmlit is designed with desktop browsers in mind.implimentaion in html5lib (originally written in python, ported to Ruby).you can use this on the web and see how the parser works there.

! Discussion

BarCamp Cambridge, Tom Morris, Semantic Web for hackers

2008-07-14T00:00:00-07:00

what's cool about microformats web?

is it the stickers?the t-shirtsthe community process

urlb.at/2f

personal information disastertravel airlines don't talk to railroads

microformats say, what problem does it solve?

perhaps there is no problem at all

what problem does blogging solve?Twitter for christ's sake?

no one knows what they do until they are popular

e.g. yahoo pipes is not practical yetit is a user experience nightmareand it doesn't have a clear defined purpose

useful becasuse there is a lot of data via rss out there

it gives us room to play

microformats is not compatible with this

should be put up data and let people play

adding some richness to the data

if if does not get used then darwin will clear up the mess

the amount of interesting data is greater than the possible number of microformats

this is a mind flip from SQL

RDF is to SQL what dynamic is to static typing

use eRDF

gives an example of sh1 of email address for putting in this data into HTML

if it maps to an RDF schema then you can use it todayif not it is based on URI's and you can make your own schema

for a free market everyone needs to take part

GRDDL only W3C could come up with a name like that

triplr.org does this

triplr.org will tell you what data your page is putting out

the semantic web is only scary if you make it scary

look at Cwm python tool, closed wold machinecan use with FOAF to make a page of hCardsyou can combine them together to see all of your friends

uses api's gives you a FOAF docuemtns and HTML page with hCard/XFN for import to for example Dopplr

tommorris.org

homework:

add some rdf data to your sitegetsemantic.com

BarCamp Cambridge, ARM microcontroler

2008-07-14T00:00:00-07:00

ARM microcontroler dev guy.

hard to use these processors, whanted to make something like this availableto normal peoplewant internet bluetooth connected devices

so they built something

he just plugged in a microcontoler with a wireless sensorhis machine things that it's a flash drive

he dregged over a binarythe device started blinking, this is the hello world of hardware hacking,cool

what can you do now? well cool stuff obviously!

the other cool thing is that there is a compiler on a websiteso you can talk about things in the context that people imagine themthis is built on top of c++

you can save straigt onto the device, as the computer just thinks that its ahard drive.now he has a flashing light on this

it's about giving people confidence in the tools that they are working with.there was no software that needed to be installed, reducing the chainbefore you get a response.

if you have a long chain, compiling and so forth,by the time you get a response your confidence that you have done the rightthingcan be low.

low chain, high level of confidence.

he then hacks the light to flash at a vairable rate depending on how totwist a switch. this normally takes about two days to get workingin the usual embeded programming systems.

the difference between this and lego mindstorms is you make this system doanything you wantand it can talk to the internet.

hacked with a gps sensor in his garden, and it told him that he was 2 milesaway from his gardenvia google-maps. It was outputting in degrees and minutes but needed to bein digital.

how does it compare to sunspots?

the main audience for this is people who want to add some control the theirdesign, but that it is not their core competence.For people who want to bridge the physicall world and the internet world.

could put an accelerometer in a rocket, and fly it for school kids. Change'fly' a rocket, and it's cool, to 'fly a rocket, and learn aboutacceleration'.

Are Academics Prostitutes

2008-07-14T00:00:00-07:00

A blog post from AcademicProductivity trundled across my reader this morning in which the blogpoints to a paper in which the author claims that academicsare prostitutes because they modify their papers in response to thedemands of reviewers. I got a sense frisson when I read the blog post andsome of the text that is quoted there from the paper, but then I read thecomments to the blog post, and they are very good at, if you like, pointingout that the paper is a bit weak in it's premise. OK, so I didn't read thepaper, but it seems to presuppose that all editorial boards work in the sameway and all reviewers and equally mendacious. Academic publishing is anactivity of a community and the reviewing process, whatever it's faults, ispart of that conversation. To point out the faults in such a way as thispaper does seems like complaining about norms that are accepted and areperhaps not actually as bad as initially laid out.

An idea for an interface for picture search

2008-07-14T00:00:00-07:00

A few weeks ago I was a beer festival in Munich, and I was talking tosomeone who was working on something, hey it was a beer festival, but then Ihad an idea about the type of interface that might be really useful fordoing search. We now search for images using text, so somewhere someone hasto build the text to image representation of the images in their database.Arguably this is one area where tagging has made an enormous improvement,ala flickr, but it seems to me that if you could make an interface thatwould allow a user to either draw or assemble a rough draft of the imagethey were searching for then this might offer a powerful and complimentaryapproach. I envisage a system where people would be able to brig together acollage of elements, a car template that could be sized, coloured, a genericperson or child object. By arranging these elements on the search interfacethe seeker could give a lot more information in their search, vis a vis thespatial arrangement of elements.

Well this I heard that there are some people already working on this, Iguess that's not surprising, I hope that they can make it work, because itwould be really cool.

tags: search, image, tech

Volvox

2006-06-29T00:00:00-07:00

“Volvox”:img:img:http://www.mulvany.net/photo/blogPics/volvox

The very first thing to die on it’s own, without the intervention of an outside agent, was an organism called Volvox. The first life that we know of whose cells perish from old age, rather than misfortune. Before humble Volvox all other living things would go on forever, the immortal bacteria, the barely describable virus. Volvox introduced death by natural causes to the world, and all living things that came after that were as complex, or more so, followed suit, and found programmed within them some pace maker, set to expire after they had done their duty on the earth. All plants, all animals that sprung from this little algae shared this little death in common with it, and also the other innovation introduced into the vaults of creation by this tumbling capsid, the ability to combine genetic material from two parents to create a truly new individual, the desire to have sex.

The tale of Adam and Eve bringing death and corruption into the garden of eden at the same moment as shame seems to me now to be more than an idle moral play. In the evolution of life on this planet sex did herald death, but the garden then was a poor and simple place before these drivers of change, of need, took hold and drove the systems of life into a perpetual war for dominance of resources that somehow and accidentally spat us breathing and panting onto the shore.

When I was younger I should have felt immortal, but there was always a fragility to the world around me. At any moment a sudden shift might take hold and break down the walls that held together what I thought of as my existence. Sitting in the sun looking down at the ants threading between blades of grass my brow would tighten as I tried to make scrutable their encoded wanderings.

And now I look about me, and it seems that those same blades of light that made clear to me my ignorance when I was a child are now highlighting again the fragility of the world, and the incredible insouciance that other people tread through their lives with.

We are all rolling along on our own hidden roller-coaster ride.

In the dark Volvox keeps rolling on, spawning it’s children, suspended in waters, like some tiny sun cast adrift in the intergalactic spaces, the centre of it’s own epicycle, it’s own birthings and dyings.

I am four billion years removed from this speck, and at the same time i could hardly feel closer.

Economics and The New World Order,

2006-06-26T00:00:00-07:00

Economics and The New World Order,

I had a rather detailed and engaging conversation on Friday night
with a couple of good friends of mine about Economics, and it's
explanatory power. My friend Brain holds that economics explains
everything, except perhaps for the laws of physics. Now I am
also tempted to buy into his arguments, he is quite a persuasive fellow,
however I intrinsically feel some caution in doing so, and I have
been trying to figure out why.

I am not sure whether my arguments hold any weight to them, or
whether I am just being an objectionable fucker for the sake of it,
but I'll try to lay out my thoughts here in any case. If nothing
else, it should draw a comment from Brain pointing out the error 
of my ways.

I think the argument is that if we want to find an explanation 
for all of the events in History, then the reason that 
some things happened, and others didn't is that the events
that occurred, and were successful, were the ones that had
an economic advantage over the ones that didn't. The scope
of what we consider, for this argument, to be economic advantage
is quite broad. Really, it seems to be anything that gives the
actors involved some sense of value. That sense can come from
moral well being, from money, from realization of religious
justification, I think. Certainly it's not simply money
as money is just one common abstract representation of
value. 
 
Now I think that I have two objections to the economic argument.

The first is that it might not have any predictive power. It 
may be tautological to say that when things happen, then
people to whom they happen feel value,  therefore those things
are the best fit things to happen (oh dear, that's not very clear
at all, but this is only a blog, and so in the spirit of such
I am not going to worry at all about not making sense). If you
can back fit all circumstances to your explanation, either
your explanation is very good, or it is very bad. If it adds no
predictive power, then it's not very good. I fear that though the
spirit of taking a truly market capitalist attitude might be 
correct, it might also have no value. In a broad market place 
there are many ideas that compete, but there are also 
a huge number of factors that go to influence success. The landscape
in which events operate (and by landscape here I mean every 
event connected to the event under consideration that has some 
effect on the event, weather, personal motivation, war, time
of year, news, car accidents, and so forth). If one's position
in the landscape has a great deal to determine whether the outcome
of an event will be a success, then though you could argue that the
even succeeded because economically it was best fit for it's 
location, I think there is something wrong with that argument.
I think that what you are looking at in economic terms is which 
direction in the landscape you want to be heading in, however
actually looking at the local geography of this abstract space
may tell you more about whether you will succeed than by looking
at where you want to go. It is not inconceivable that the 
only thing that matters is your position in this space, and 
not by any means your intent. The purely economic argument would then 
be a comforting chimera that we can wrap around ourselves to 
protect us from the vagaries of the world.

That's all well and good, but is there any way to tease these idea
apart and to test them? it's highly unlikely, but there may be some
option not from looking at the world, but by looking at worlds that
we create. In a simple model the economic argument should work. 
There are many studies of what is known as the prisoners dilemma,
I don't know much about what the studies say, but I think 
they should be a good test of economic predictive theory.
Therefore in a simple game economics should have a high 
predictive power to give a winner. Add complexity to the 
game and see how predictability changes as a rate of increasing
complexity. Hmm, actually, the stock market is a very good 
example of doing something like this, but we can't reverse 
engineer the world to see if we can make it more predictable.

Well, that was part one of my argument, which even I have to admit
just comes down to saying "perhaps things are just a bit more complicated",
and since we all seek simplicity, then perhaps there is not 
much harm in looking to economics (other than being wrong an awful
lot, and retroactively fitting by saying, ahh, but of course we didn't
factor in economic driver blah blah). Still, it's in our
nature to be wrong loving mammals, so I shouldn't worry too much.

The next thing that unsettles me is even if the economic argument
it is wright, and it would be quite nice if it were right, for if
it were it would give us the true power to change the world, 
there may be a temptation to confuse correctness of explanation
with justification for action. Being Irish I have a built-in
distrust of a lazzaiz-fair market, as this was the decision
that contributed to the deaths of many Irish people during the 
great famine of the 1840's. By saying that the open market
is the best option we forgo our ability to intervene when there
is pressing reason to do so. The best open market for ideas
is evolution, but even in evolution market forces can press
species into extinction, though sexual over-selection. The
same may be happening with America where there is a strong
market drive for light entertainment, cheap credit and cheap oil.
Perhaps these drivers are acting locally and may not be the
best actions that the market could be taking locally. Even though
humans are subject to evolution I believe strongly that we have
the capacity to transcend evolution (that sounds wishy-washy, but
I mean that we have the moral and technological ability to make decisions
that are nut just driven by competition with other species, indeed
our ability to do so almost gives us a moral obligation to do so, 
and that would be fine, as soon as we figure out what morals are).

Well, I have more to say, but for the time being that is enough on this topic 
from me.

Economics and The New World Order,

2006-06-26T00:00:00-07:00

Economics and The New World Order,

I had a rather detailed and engaging conversation on Friday night
with a couple of good friends of mine about Economics, and it's
explanatory power. My friend Brain holds that economics explains
everything, except perhaps for the laws of physics. Now I am
also tempted to buy into his arguments, he is quite a persuasive fellow,
however I intrinsically feel some caution in doing so, and I have
been trying to figure out why.

I am not sure whether my arguments hold any weight to them, or
whether I am just being an objectionable fucker for the sake of it,
but I'll try to lay out my thoughts here in any case. If nothing
else, it should draw a comment from Brain pointing out the error 
of my ways.

I think the argument is that if we want to find an explanation 
for all of the events in History, then the reason that 
some things happened, and others didn't is that the events
that occurred, and were successful, were the ones that had
an economic advantage over the ones that didn't. The scope
of what we consider, for this argument, to be economic advantage
is quite broad. Really, it seems to be anything that gives the
actors involved some sense of value. That sense can come from
moral well being, from money, from realization of religious
justification, I think. Certainly it's not simply money
as money is just one common abstract representation of
value. 
 
Now I think that I have two objections to the economic argument.

The first is that it might not have any predictive power. It 
may be tautological to say that when things happen, then
people to whom they happen feel value,  therefore those things
are the best fit things to happen (oh dear, that's not very clear
at all, but this is only a blog, and so in the spirit of such
I am not going to worry at all about not making sense). If you
can back fit all circumstances to your explanation, either
your explanation is very good, or it is very bad. If it adds no
predictive power, then it's not very good. I fear that though the
spirit of taking a truly market capitalist attitude might be 
correct, it might also have no value. In a broad market place 
there are many ideas that compete, but there are also 
a huge number of factors that go to influence success. The landscape
in which events operate (and by landscape here I mean every 
event connected to the event under consideration that has some 
effect on the event, weather, personal motivation, war, time
of year, news, car accidents, and so forth). If one's position
in the landscape has a great deal to determine whether the outcome
of an event will be a success, then though you could argue that the
even succeeded because economically it was best fit for it's 
location, I think there is something wrong with that argument.
I think that what you are looking at in economic terms is which 
direction in the landscape you want to be heading in, however
actually looking at the local geography of this abstract space
may tell you more about whether you will succeed than by looking
at where you want to go. It is not inconceivable that the 
only thing that matters is your position in this space, and 
not by any means your intent. The purely economic argument would then 
be a comforting chimera that we can wrap around ourselves to 
protect us from the vagaries of the world.

That's all well and good, but is there any way to tease these idea
apart and to test them? it's highly unlikely, but there may be some
option not from looking at the world, but by looking at worlds that
we create. In a simple model the economic argument should work. 
There are many studies of what is known as the prisoners dilemma,
I don't know much about what the studies say, but I think 
they should be a good test of economic predictive theory.
Therefore in a simple game economics should have a high 
predictive power to give a winner. Add complexity to the 
game and see how predictability changes as a rate of increasing
complexity. Hmm, actually, the stock market is a very good 
example of doing something like this, but we can't reverse 
engineer the world to see if we can make it more predictable.

Well, that was part one of my argument, which even I have to admit
just comes down to saying "perhaps things are just a bit more complicated",
and since we all seek simplicity, then perhaps there is not 
much harm in looking to economics (other than being wrong an awful
lot, and retroactively fitting by saying, ahh, but of course we didn't
factor in economic driver blah blah). Still, it's in our
nature to be wrong loving mammals, so I shouldn't worry too much.

The next thing that unsettles me is even if the economic argument
it is wright, and it would be quite nice if it were right, for if
it were it would give us the true power to change the world, 
there may be a temptation to confuse correctness of explanation
with justification for action. Being Irish I have a built-in
distrust of a lazzaiz-fair market, as this was the decision
that contributed to the deaths of many Irish people during the 
great famine of the 1840's. By saying that the open market
is the best option we forgo our ability to intervene when there
is pressing reason to do so. The best open market for ideas
is evolution, but even in evolution market forces can press
species into extinction, though sexual over-selection. The
same may be happening with America where there is a strong
market drive for light entertainment, cheap credit and cheap oil.
Perhaps these drivers are acting locally and may not be the
best actions that the market could be taking locally. Even though
humans are subject to evolution I believe strongly that we have
the capacity to transcend evolution (that sounds wishy-washy, but
I mean that we have the moral and technological ability to make decisions
that are nut just driven by competition with other species, indeed
our ability to do so almost gives us a moral obligation to do so, 
and that would be fine, as soon as we figure out what morals are).

Well, I have more to say, but for the time being that is enough on this topic 
from me.

Partially Attended

Mendeley, a investment branch of the citation bank.

SOPA and PIPA stink, but the RWA is more dangerous to science.

Climbing Goals 2012, review 2011

2011 hi-lights:

Goals for 2012 are as follows:

example title

Update on MathJax

Converting between dates and unix time in Python

Going from a date to a unix time:

Going from a unix time to a date:

Nature, Whiskey and me

Hindawi have an awesome reviewing system.

Megajournals

The rude health of Open Access Publishing.

SOLO11, day1, morning sessions.

Session on engaging with peer review

NPG API session

Science Online London Keynote, Michael Nielson on Open Sciecne

Michael Nielson Keynote on Open Science

Talkfest, science and community

Introduction

Are community groups based around hobbies a good way to reach new audiences?

Galaxy Zoo

Linda talking about the OPAL project (I don’t know anything about this)

Stephen Curry - blogger!

QA

ArXiV at 20, a brief review.

There are real costs associated with running services on the web.

What we can do on the web has been well imagined for a long time, we just don’t seem to be doing it.

In spite of researcher anxiety, these tools bring real value.

Serendipity, a chance encounter (obviously)

Robert Bunsen's Birthday

Sits meeting notes, November 2010.

Some topics from the last meeting

Some potential topics for this meeting

Actual topics discussed

topic discussions

Authentication

takeaways from this talking point:

Linked data

some ontologies that might be useful

Action Items

Web Archiving

Issues:

Action Items

Lunch

Identifiers for people

Resources

Microservices

Resources

SEO and search engine optimisation

Resources

Actions

Lightweight Languages

Resources

mini-review of "Measuring the User Experience on a Large Scale"

Our gospels define us

Rat in the Snow

Turning the Physics ArXiV into an Open Peer Review System.

Cancelled :( Drinks with Chris Wiggins.

CANCELLED

Wrangling image metadata

Video workflow with a mac using iMovie and capturing with a Lumix GF1

San Francisco, and science hack day discussions

DLF opening keynote, beyond buckets and boxes

Beyond buckets and boxes.

General overview

Coherence of design

clr mission

The role of DLF

Arriving in Paulo Alto for the DLF fall forum

save mailfolder messages to a file using applescript

Citation Style Language

The Future of Knowledge Organisation on the Web, a one day conference.

Nigel Shadbolt on Government Linked Data, a Tipping Point for the Semantic Web

Antoine Isaac, SKOS and Linked Data

Ricard Wallis from Talis on The Linked Data Journey

Steve Dale, Linked Data in Local Government: The Knowledge Hub

Prof. Dr. Martin Hepp, Linked Data in E-Commerce, The GoodRelations Ontology