February 2008 – howsheilaseesIT

A capital day for assessment projects

Last Monday CARET, University of Cambridge hosted a joint workshop for the current JISC Capital Programme Assessment projects. The day provided an opportunity for the projects to demonstrate how the tools they have been developing work together to provide the skeleton of a complete assessment system from authoring to delivery to storage. Participants were also encouraged to critically review progress to date and discuss future requirements for assessment tools.

Introducing the day Steve Lay reminded delegates of some of the detail of the call under which the projects had been funded. This included a focus on “building and testing software tools, composite applications and or implementing a data format and standards for to defined specification” – in this case QTI. The three funded projects have built directly on the outcomes of previous toolkits and demonstrator activities of the e-framework.

The morning was given over to a demo from the three teams, from Kingston, Cambridge and Southampton Universities respectively, showing how they interoperated by authoring a question in AQuRAte then storing it in Minibix and finally delivering it through ASDEL.

Although the user-interfaces still need a bit of work, the demo did clearly show how using a standards based approach does lead to interoperable systems and that the shorter, more iterative development funding cycle introduced by JISC can actually work.

In the afternoon there were two breakout sessions one dealing with the technical issues around developing and sustaining an open source community, the other looking innovations in assessment. One message that came through from both sessions was the need for more detailed feedback on what approaches and technologies work in the real world. Perhaps some kind of gap analysis between the tool-set we have just now and the needs of the user community combined with more detailed use cases. I think that this approach would certainly help to roadmap future funding calls in the domain as well as helping inform actually practice.

From the techie side of the discussion there was a general feeling of there still being lots of uncertainty about the development of an open source community. How/will/can the 80:20 rule of useful code be reversed? The JISC open source community is still relatively immature and the motivations for be part of it are generally because developers are being paid to be part of it – not because it is the best option. There was a general feeling that more work is needed to help develop, extend and sustain the community and that it is at quite a critical stage in its life-cycle. One suggestion to help with this was the need for a figure head to lead the community – so if you fancy being Mr/Mrs QTI do let us know:-)

More notes from the day are available for the projects’ discussion list.

Assessment, Packaging – where, why and what is going on?

Steve Lay (CARET, University of Cambridge) hosted the joint Assessment and EC SIG meeting at the University of Cambridge last week. The day provided and opportunity to get an update on what is happening in the specification world, particularly in the content packaging and assessment areas and compare that to some really world implementations including a key interest – IMS Common Cartridge.

Packaging and QTI are intrinsically linked – to share and move questions/items they need to be packaged – preferably in an interoperable format:-) However despite recent developments in both the IMS QTI and CP specifications, due to changes in the structure of IMS working groups there have been no public releases of either specifications for well over a year. This is mainly due to the need for at least two working implementations of a specification before public release. In terms of interoperability, general uptake and usabillity this does seem like a perfectly sensible change. But as ever, life is never quite that simple.

IMS Common Cartridge has come along and has turned into something of a flag-bearer for IMS. This has meant that an awful lot of effort from some of the ‘big’ (or perhaps ‘active’ would be more accurate) members of IMS has been concentrated on the development of CC and not pushing implementation of CP1.2 or the latest version of QTI. A decision was taken early in the development of CC to use older, more widely implemented versions of specifications rather than the latest versions. (It should be noted that this looks like changing as more demands are being made on CC which the newer versions of the specs can achieve.)

So, the day was also an opportunity to reflect on what the current state of play is with IMS and other specification bodies, and to discuss with the community what areas they feel are most important for CETIS to be engaging in. Profiling did surface as something that the JISC elearning development community – particularly in the assessment domain – should be developing further.

In terms of specification updates, our host Steve Lay presented a brief history of QTI and future development plans, Adam Cooper (CETIS) gave a round up from the IMS Quarterly meeting held the week before and Wilbert Kraan (CETIS) gave a round up of packaging developments including non IMS initiatives such as OAI-ORE and IEEE RAMLET. On the implementation side of things Ross MacKenzie and Sarah Wood (OU) took us through their experiences of developing common cartridges for the OpenLearn project and Niall Barr (NB Software) gave an overview of integrating QTI and common cartridge. There was also a very stimulating presentation from Linn van der Zanden (SQA) on a pilot project using wikis and blogs as assessment tools.

Presentations/slidecasts ( including as much discussion as was audible) and MP3s are available from the wiki so if you want to get up to speed on what is happening in the wonderful world of specifications – have a listen. There is also an excellent review of the day over on Rowin’s blog.

LETSI update

Alongside the AICC meetings in last week in California, there was an ADL/AICC/LETSI Content Aggregation Workshop. Minutes from the meeting are available from the LETSI wiki. There seemed to be a fairly general discussion covering a range of packaging formats from IMS CP to MPEG 21 and DITA.

As we have reported previously, the ADL would like to see a transition to a community driven version of SCORM called core SCORM by 2009/10. This meeting brought together some of the key players although it looks like there was no official IMS representation. It does seem that things are still very much at the discussion stage and there is still a way to go for consensus on what de jour standards core SCORM will include. There is another LETSI meeting in Korea in March, before the SC36 Plenary Meeting. One positive suggestion that appears at the end of the minutes is the development of white paper with a clear conclusion or “call to action’. Until then it’s still difficult to see what impact this initiative will have.

OpenLearn XML processor update

At the CETIS conference last November, one of the many highlights of Tony Hirst’s presentation was the openlearn XML processor which disaggregating the various parts of an openlearn unit (text, video, audio etc). Tony has just posted an update on some further work he has been doing with the processor. Well worth a read as it gives a good overview of how the processor works, and thoughts Tony has for future development including representations of course designs.

What does HE want from publishers?

Whilst colleagues were at the MDR SIG meeting on Tuesday, I was in another room in London at the annual Publishers Association conference to see what the answers(s) to the above question might be. Books, was the answer and I guess a simplistic, semi-accurate summary would be “anything else we can get for free” – but this has to be qualified by meaning free at point of access.

Actually I was quite depressed at various points during the day. Not least the at the start when we were shown part of a video vox-pox of students and staff at Manchester discussing how, why, where and when they used textbooks. We were then told we could buy copies of the DVD for Â£15 – they had some there and a form we could post back to them. Why weren’t the PA putting chunks of this on their website, YouTube and/or TeacherTube? That just seemed to me to encapsulate the differences between educators and publishers – particularly those of us interested in producing and sharing learning materials.

Perhaps I was a bit of fish out of water in that the majority of the audience were librarians and or publishers. But I don’t really find listening to someone lamenting over what a terrible breed of people his generation has created now that “four out of five (music) downloads are illegal”, particularly inspiring, helpful or even controversial.

However on a slightly more positive note, there did seem to be a recognition of a need for changing business models to allow the development of text books/ebooks which met the changing needs of educators, students and the publishing industry. However there was no concensus as to what form any such model would take.

There were interesting presentations from the OU about OpenLearn and from the JISC ebooks observatory project. It is looking like the ebooks observatory is going to provide significant data on use of ebooks as already they have had nearly 20,000 responses to their initial baseline survey. However both these projects although providing access free at point of use, rely on not insignificant funding which is (probably) not sustainable. Whilst there have been great strides in opening access to e-journals the same can’t be said for ebooks or learning resources. How we can learn lessons from the former really didn’t get addressed yesterday.

Latest poll results

As a follow up to the recent JISC one day conference “Using Learning Resources: Transforming the Educational Experience”; I thought it would be interesting to see what the SIG thought of some of the common themes coming through from the day about potential areas for funding and what the community thought priorities should be. Once again there was a great response to the poll – so thank you if you voted.

Perhaps unsurprisingly developing more user friendly tools for creating and sharing learning resources was the clear winner with 63% of the votes. I think this reflects how much people in the SIG just want to get on and develop more ways to create and share – in particular activities, designs and assessments. This contrasts with the more strategic views coming through at the event where discussions around engaging middle management through developing business models and providing clear IPR/copyright guidance were coming to the fore.

The results were as follows:

*developing more user friendly tools for creating and sharing learning designs 63% (27 votes)
*developing more ‘open’ approaches eg a JISC equivalent of OpenLearn 19% (8 votes)
*developing use cases for middle management 5% (2 votes)
*developing clear IPR and copyright guidance 5% (2 votes)
*Other: open call for evaluation and research projects; Re-using and rejuvenating existing resources; Re-use/Rejuvenation of existing content; Finally doing something for FE
(43 votes in total)

More information about this poll is available from the EC SIG wiki.