A new year, a major update

Now that the JISC project has come to an end, and our busy term is over with, we thought that we would continue using the blog to document our progress in implementing Primo at Coventry University Library.

Towards the end of last term things progressed rapidly with 'Locate' as we moved towards a soft-launch date of 11th January 2011. Some of the things we worked on included:

• Creating a front page for Locate using the feedback and results from the focus group undertaken earlier in the year - the aims were to make sure it wasn't too 'wordy' and to have links out to further information rather than have it all on the page. At the moment there is quite a lot of 'explanation' on the page to let end-users know what Locate is, and what it is replacing. We hope to remove some of this as the product becomes more integrated into the library. The focus group also suggested that they would like links to other support services and we have managed to incorporate these in to the front page in the boxes handily provided in the original Primo front page template.

Locate Front Page

• We have made small changes within the availability/holdings lists to make sure that they looked clean and clear for end-users. This involved quite a bit of trial and error, but eventually we managed to get to a screen that provided essential details in the brief results view, and more detailed information in the larger full-screen' view. This was tricky as we had to consider all resource types, and some terminology doesn't fit across journals, books and media items consistently! 

• We have also worked on creating an 'OPAC-version' of Locate - which for us means adapting the interface so that it can be used on stand-alone OPAC machines on the subject floors so that both students and non-university members can search the library's holdings. This has involved a lot of work in 'locking-down' the machines and Locate so that access to external resources e.g. e-books and e-journals is restricted. We are almost there with the OPAC version, but we still have a ew things to work on before we release it to the students. Hopefully there will be one 'trial' Locate machine alongside our original OPAC over the next few months so that students can get a taster of what it will be like when we switch fully in the Summer.

OPAC Front Page

 

We hope to continue posting updates over the next few months, including a link to the live version of Locate. We are also hoping to do some testing of Primo Central, and to gauge feedback from staff and students to Locate and Primo Central, which we can then use to make modifications before the launch in the Summer.

Final blog post - appendix

By popular demand, here are the requested screen dumps...

1. Locate interface displaying search results at Coventry University




2. Locate displaying integrated item availability information at Coventry University



3. Sample user reservation display at Coventry University

COVPRIMO Final Blog Post

Background
Coventry University is a post-1992 University with around 14,000 FTEs. The student population of undergraduates and taught postgraduates is very diverse studying across a wide range of subject areas and located in five Schools and Faculties (Art and Design, Business Environment and Society, Engineering and Computing, Health and Life Sciences and Lifelong Learning). The emphasis is on teaching, with a strong commitment to e-learning complimenting face to face interaction in the classroom. Research activity is also important.

The Lanchester Library is a busy academic library, often with over 5,000 people passing through the gates each day. Around 350,000 items are stocked and over 500,000 issues / renewals are made per annum. Around 12,000 new items are purchased annually. Over a million full-text journal articles are downloaded each year and there are over 800,000 e-book accesses.

Intended outcome(s)
The main objective of the project was to implement Ex Libris’s Primo discovery tool as a single user interface for Coventry University’s users to make the starting point for all their Library searches. This should help to provide a more consistent user experience than searching the Library Catalogue, MetaLib (our federated search tool) and Equella (our institutional repository) individually. This should help to improve search efficiency and so increase the use of our e-resources.

The challenge
In a context in which the ‘information seeking behaviour’ was perceived to be changing and the expectation of intuitive and effective interfaces was emerging, we were conscious of the need to enhance our resource discovery environment.

It was felt that the Aleph Library Catalogue was outdated and had none of the common web 2.0 functionality that our users expected to see. We had developed it as much as we possibly could, introducing a pretty major overhaul 2 years ago. This was carried out locally by our Library Systems team following some usability testing and was a definite improvement. However there were many other changes that we wanted to introduce that just weren’t possible within the constraints of the Aleph OPAC set-up.

Another concern came from the fact that we were using MetaLib alongside the Aleph OPAC as the main route to access our e-resources. This meant that users had 2 different places to search for Library content, and sometimes lead to confusion over which interface they were searching and which one they should be using. Usability testing of MetaLib was carried out at the same time as the OPAC usability tests and revealed that it wasn’t as intuitive as we would have liked. Added to this we are also aware of the growing numbers of international students studying at Coventry. Our understanding from focus groups is that they tend to come to do postgraduate courses with no previous experience of e-resources and do not find MetaLib intuitive.

Authentication to the OPAC and to MetaLib is carried out using 2 different methods, meaning that users are required to login with different usernames and passwords. OPAC authentication is by barcode from the user’s University card, whereas MetaLib uses LDAP to authenticate with the person’s University username and password. We were also aware from responses to the questions in our ‘user satisfaction surveys’ relating to ease of access to online resources that this was an area which we could improve on.

As a result of this it was decided that a new interface that combined the functionality of both the OPAC and MetaLib, was authenticated using Shibboleth and that was very configurable would be a good investment.

Established practice
The established practice was using the Aleph OPAC to search for books and journals but using MetaLib to search for databases by name or by subject. MetaLib also has a Find e-Journal option. This displays holdings information about e-journals which have been activated in the SFX link resolver. If an e-journal is available and displays in this list a link will also be provided back to the OPAC if the journal is available in print. However, if a journal is only available in print this will not display in the Find e-Journal list within MetaLib. This will only be discoverable from the OPAC, although SFX will provide links to print only journals if an article from one of these titles is found in a database.

The LMS advantage
The introduction of Primo as a replacement for the OPAC and MetaLib should simplify any confusion over which interface users should be searching. It is also designed in a much more intuitive way and works like many other popular websites. During our usability testing favourable comparisons were made to Tesco Online and Amazon. Primo is also much more configurable than the Aleph OPAC and than MetaLib. This enables us to customise it much more easily, something that we have already done based on the usability testing that we carried out as part of this project.

The usability tests that we ran as part of the project were extremely helpful. They enabled us to identify aspects of the out-of-the-box Primo which we wanted to change, before the launch. The testing was also of interest to Ex Libris and we had a conference call with the Primo Product Manager in September. This enabled us to feed back the main issues which we believe our usability tests revealed. This included things which we weren’t currently able to change and Ex Libris are going to consider introducing these as future updates to the Primo product.

We are also introducing Primo Central, which provides pre-harvested and indexed metadata from a range of databases. This enables the user to quickly retrieve journal articles alongside their other content (books, journals, videos, DVDs, databases, etc.). Previously to search for journal articles a federated search would need to be done, which carries out a live search of selected databases, so tends to be slow. Primo Central avoids this problem as the metadata is already harvested. We are just in the process of introducing Primo Central, so hope to carry out more analysis of its functionality over the next few months.

Another benefit is that Primo is being configured to authenticate via Shibboleth. This means that there will be no confusion about what credentials users need to login. It will also have an additional benefit for users who login to Primo and then link to a Shibboleth-authenticated database. These users will not need to login to the database during the same session, as they will have already been authenticated.

Key points for effective practice
Key points to note for other practitioners are as follows:
The timescale we were working to for implementing Primo was rather ambitious. We were trying to introduce it before the end of August, which is what would have been required to allow Subject Librarians to prepare their teaching materials for the new term. This might have been possible if we hadn’t wanted to carry out the usability testing and make significant changes to the Primo front end. However, as our Primo installation was delayed until July, with our first training day taking place on July 22nd, we weren’t able to carry out the testing until the final week of July. We did make excellent progress in customising the front end throughout August, thanks to the hard work of our Ex Libris UK contact who helped us make numerous changes until his deserved vacation in the 2nd half of August. Unfortunately, while he was away there wasn’t much additional support available meaning progress was hindered.

A number of other major concerns meant that the launch of Primo had to be postponed anyway. The main one of these was the setting up of Shibboleth authentication, which required involvement from the Library Systems Team, our IT Services department and one of Ex Libris’s German offices. This work began in August and continued throughout September, and there was no possibility of going live until this had been introduced. The delay was made worse by the fact that IT Services were migrating from Shibboleth 1.3 to 2 in early September

The other really big change which we were waiting for was the introduction of Primo Central. This was being rolled out to Primo customers over the summer, but wasn’t available to us until October. Given that adding this was a major change in the way Primo functioned it seemed sensible to delay the launch until after this had been added and configured to our specifications.

Other problems encountered during the project were the technical challenges of the transfer of data from Aleph and Equella (the institutional repository). It was also difficult to obtain advice on suitable backup routines from Ex Libris.

Another issue was experienced around the installation of Syndetics ICE for indexed content enrichment. We were told by Ex Libris that Primo was compatible with ICE and we were keen to test this functionality. However, when we tried to arrange this we found out that Primo wasn’t currently working with ICE. We were told it wasn’t a major problem, but we’re still waiting for this to be set up.

From a communications point of view it wasn't always easy discussing involved / complex technical issues over the phone with our Ex Libris Project Manager who was based in Germany. It was also not clear what we could expect to be carried out by Ex Libris as part of their basic installation service. We did identify a long list of changes to the interface following our usability testing, and our Ex Libris UK contact was extremely helpful in working with us on these. However, with hindsight more detailed clarification over where responsibilities lay would have been helpful.

Conclusions and recommendations
We are happy with our Primo installation so far and keen to make it available to our users. It’s a shame that we weren’t able to release it before the start of the new academic year, but it was definitely the right decision to delay things until Shibboleth had been configured and Primo Central had been added. We will then be able to launch a beta version of Primo. We plan to display a feedback form on the site to request more input from our users. We also hope to carry out further usability testing on the customised Coventry version of Primo and on Primo Central. We will then be able to make further changes (if necessary) before launching Primo as our single user interface.

The feedback we had from our usability testers was very positive and we believe this will also be the case with our other users. Based on our experience of Primo so far we believe it will be a major improvement over the previous search experience at Coventry, particularly with the introduction of Primo Central.

Recommendations:
• Make sure you have the right mix of people on the project group to ensure that the teams most involved in using the new system can present an equal viewpoint and to ensure project ownership across different sections of staff.
• Under no circumstance underestimate the time & resource such a project requires.
• Involve external suppliers (in this case of our repository, Equella) at an early stage.
• Allow enough time to customise the front end. If you plan to make significant changes then it would be sensible to allow a minimum of 2 months, more if you are planning to carry out usability tests and need to allow time to analyse these.
• If you know you are going to be doing testing over the summer period when there aren’t many students around try to recruit testers who live locally before the end of the summer term.
• Carry out a pilot study before you do your real testing. This can help to reveal any questions that don’t work as intended and will enable you to make changes before doing the test with your volunteers.
• Make sure you involve Academic staff in your testing.
• Try to get access to a usability lab to carry out the tests and record what takes place for accuracy.
• Get feedback and buy-in from Subject Librarians and staff who work on enquiry desks as these will be the people who will be teaching students how to use the system
• Try to ensure you have accurate creation and maintenance of the Name Authority Files in your LMS. We didn’t have this in Aleph and it would have made FRBRisation much more accurate. This is something that we are now having to review so that we get one FRBR group for something instead of 2 or more.

Additional information
Project Blog: http://covprimo.blogspot.com/
Coventry University Primo installation: http://primo.coventry.ac.uk/primo_library/libweb/action/search.do?dscnt=1&fromLogin=true&dstmp=1279105993031&vid=COV_VU1&fromLogin=true
Ex Libris Primo Overview: http://www.exlibrisgroup.com/category/PrimoOverview

Primo Usability Testing - Details Post #3

This is the third of our 3 posts giving a detailed write-up of our usability testing.

Section 3 – Finding Databases and Post-Test Discussion

Question 10 - Your lecturer has told you to search the database ASSIA (Applied Social Sciences Index & Abstracts). How would you find it and access it.

In this question we were attempting to find out the visibility of the Find Databases link (as it is only shown in the Articles etc. tab), and to see how testers used the functionality within the Lightbox to access a specific database. We were aware before the testing that an alternative method of finding a named database existed i.e. searching by name in the Books etc. tab and only 1 tester used this method. All 9 other testers used the Find Databases link – 3 testers needed prompting to either go in to the articles tab, or to look around the screen to see if they could see a link. 3 testers saw the Find Databases link straight away, and 3 others saw it after moving between the Books and Articles tabs. Some testers tried to use the options within the Advanced Search in the Articles tab before noticing the Find Databases link.
In terms of actually accessing ASSIA, of the 9 testers who used the Lightbox, 5 ticked on the checkbox expecting that it would open up the database. 2 of these then went on to click on ‘Selected Databases’ in the Lightbox which automatically took them out of the Lightbox and back to the Article etc tab search screen with the selected database as a ‘quickset’. This was very confusing for the testers as there was no warning or indication of what had happened. To access ASSIA the testers needed to click on the title of the database and 4 did this straight away without ticking the checkbox, and 3 others clicked on the title after ticking the checkbox and having nothing happen.
Recommendations:
• Make the Find Databases link appear in both the Articles etc tab and the Books etc tab – Ex Libris are looking in to this.
• Make the Find Databases link more visible – some testers commented that it was hidden away in the corner – we have aligned this to the left-hand side, immediately above the main search box and used a larger font.
• Provide instructions for students on finding named databases on the front page of Primo.
• Remove the checkboxes from the Lightbox and remove un-needed functionality (e.g. ‘Selected Databases’) – Ex Libris have removed the checkboxes and have taken out some of the functionality within the Lightbox. This needs more development work in the future.

Question 11 - ASSIA isn’t finding you the articles you want and you’re not sure what other databases you should use to find articles from the Business & Economics subject area. How would you find a list of databases recommended for this subject?

In this question we were looking at whether the testers would be able to find a list of databases relevant to a specific subject area from the information available to them in the Lightbox. Many of the testers were already in the Lightbox when they attempted this question and found a list of databases relatively quickly although many clicked on several drop-down menus before realising the information they wanted was in Category – this indicates that more work needed to be done to ensure that this was more intuitive. Some testers suggested using ‘Subject’ instead of ‘Category’ to indicate more clearly. Some testers also seemed to expect the list at the bottom of the Lightbox to automatically switch to the chosen subject area and it took a few seconds for them to click on the Find Databases link instead. This expectation could be due to their experiences within Metalib where the list automatically opens.
Recommendations:
• Re-name ‘category’ as something more intuitive – this has been re-named as ‘subject’.
• Insert text in to ‘Category’ box to indicate to students that they can choose a subject area i.e. ‘Select subject area’.

Post-Test Discussion / General Observations

Many testers thought that once they were used to the new site they would find it easy to use, especially as it had everything in one place unlike the library catalogue and eLibrary. Some suggested that the new site was more professional than the other library search tools they had used.

Facets – The majority of testers thought that the facets were best placed on the left-hand side of the screen – some testers mentioned websites such as Google and Tesco in comparison. All testers thought that the facets would be useful to them when searching library resources; however some commented that work could be done to make them more visible e.g. some testers didn’t notice them until they were pointed out. It was encouraging throughout the tests to see some testers using them on searches that didn’t explicitly require them to do so.

Reviews & Tags – None of the testers knew what a Tag was. When we explained to them that they could add their own keywords to individual catalogue records and asked if they thought this was useful some of them did think it would be. However, most of them were thinking of using the Tags as a way for them to retrieve records at a later date. It would probably be easier for them to just save these records to their personal Primo space (e-Shelf or My Favourites as we renamed it). When a scenario of tagging records that could be useful for other students working on a joint project was suggested most testers thought that this was a good idea. However, the fact that most of them didn’t think of this or any other social-networking way of using tags seems to imply that this functionality may not be well used. We are planning to leave the Reviews & Tags function switched on and see how much use it gets.

Other Actions based on the Usability Tests:
• The ‘More’ tab was removed as it duplicated information found in other tabs and was felt to be irrelevant.
• The main tabs were relabelled as ‘Books, Journals, Media & More’ and ‘Articles & More’ to differentiate the types of content in each tab.
• Some tabs were re-ordered to give more prominence to actual availability e.g. the ‘Locations’ tab (re-named ‘Availability’) is now either the first or second in the list rather than appearing after the ‘Request’ tab.
• The ‘New Search’ link needed to be made much clearer – a few testers commented that they couldn’t find a way to clear their search – we have left-justified this and made the font bigger.
• The e-shelf was misleading for quite a few testers, as they thought this was an area where they could search for e-resources – we have now re-named this as My Favourites in keeping with the general trends found on other mainstream websites.

Final technical observations

For those interested in a quick summary of the technical aspects of this project, the implementation notes which form part of the 'one minute blog' post may prove useful.

Here I thought I'd concentrate on a couple of technical areas that have eaten away at the time.

Shibboleth implementation is one of the main culprits in this category. Although this formed part of the contract with our supplier, there were still significant obligations, both for library systems staff as well as IT Services staff. The university already had a Shibboleth idp service in place but on the library side we started from scratch and encountered a fairly steep learning curve. There's a whole new vocabulary to learn when addressing issues that surround authentication - it's wasn't immediately obvious where to go for help. Fairly early on our PM, sporting a large grin, pronounced, "I don't do Shibboleth!" I can see why. Added to this, the upgrade to the unversity's idp from Shibboleth version 1.3 to version 2.0, right in the middle of our testing phase, set us back considerably.

Following on from the Shibboleth issue, we were also aware we needed to upgrade the OPAC authentication functionality which still made use of our 14 digit patron barcodes. In order to move to Shibboleth authentication, it would prove necessary to insert patron usernames into their Aleph records. Inserting this data into the relevant field (z308_rec_key) with the appropriate flag would prove time-consuming. Occasionally the unversity issues a new username to a user - dealing with updates of this type continues to create some problems.

Other time-consuming activities included establishing data export / import routines for the source systems. The library management system Aleph proved troublesome - plenty of room for improvement in the supplier's documentation here. Having said that, the work our PM carried out on the university's repository product, Equella, proved invaluable.

Finally I'm sure my colleague who worked through many of our technical troubles (and is currently taking a well-earned holiday) would want me to mention the unavoidable overhead that is incurred when installing separate third party products which all go to form part of the complete installation.

By definition any summary such as this will struggle to give a complete picture; I hope these observations have at least provided a useful snapshot.

Primo Usability Testing - Details Post #2

This is the second of our 3 posts giving a detailed write-up of our usability testing.

Section 2 - Searching for known journals and journal articles

Question 7 - Your lecturer has told you to read an article in the most recent issue of the journal, the Harvard Business Review. Check to see if this is available in print, electronically or in both formats.

In this task we wanted to see how the testers would search for an individual journal. There are clear options for this in the library catalogue and eLibrary; however Primo does not explicitly offer an option for this at first glance. Only 2 of the 10 testers were able to correctly identify that we had both print and electronic access to the journal. The majority of testers saw ‘Online access’ and ‘Media format: E-Journal’ and suggested that we only had electronic access to the journal. Only two testers clicked on the Locations tab which is where they would find details of the print access. One tester found the print access by clicking on the ‘More’ tab and noticing the ‘Check print availability’ option in the SFX menu – however they did not verbally confirm if there was electronic access. 4 of the testers tried to search for the journal in the ‘Articles etc’ tab as they assumed that this was where they would find journals. Some of these were prompted to look at the ‘etc’ part of the tab name – one tester stated that they thought this was very unprofessional terminology to use in a higher education environment. Two of the testers used the Advanced Search option in the Books etc tab and chose ‘Journal’ from the format. Other testers commented that they had previously used the Journal Search or Find e-journal options with the Library Catalogue and eLibrary. This indicates that the testers were looking for ‘somewhere’ to search for journals and wouldn’t necessarily use the Books etc tab for this purpose.
Recommendations:
• Re-name the ‘Books etc’ tab to include the word ‘journal’ and create a more professional-sounding name – this would indicate to users more clearly where they should search for different material types – this has been re-named ‘Books, Journals, Media & More’.
• Find a way of making print journal access more visible to the user – possibly re-naming the Locations tab could help with this.
• Check with Ex Libris to see if different text can be displayed in the ‘Availability Status’ line (would also help with e-books – where print availability is also hidden) – Ex Libris responded that a different text can be displayed – we are still looking in alternative wording.
• Change the information in the Media Format section of the ‘Details’ tab to include both print and electronic access – if this is not possible would removing it altogether be an option? – we have removed the Media format option from this tab.

Question 7a - What years are available and from where?

Out of 9 testers who attempted the question, only 2 were able to correctly identify the years of access as from 1952 (print) and from 1922 (electronically). The majority of testers thought that we only had access to one year of the journal – 1922. This date was mostly identified in the results list, the details tab or in the sfx menu. 1 tester did not verbally conform any dates at all, despite scrolling through a list of journal issues in EBSCO. Unfortunately, due to the lack of recognition of print availability (as shown in question 7) it was not possible to effectively evaluate the information shown in the journals locations tab, although the testers who found it were confused by the amount of information on the screen and took some time to notice the Summary Holdings statement showing the years available.
Recommendations:
• If possible include the journal ‘from and to dates’ in the brief display and full details tabs (not possible due to cataloguing implications)
• If the above is not possible, then remove the date entirely from these sections as it could be deemed to be very misleading – we have removed the date for journals.
• Try to remove all of the information in the journals locations tab to make the Summary Holdings statement stand out more prominently (unfortunately this is unable to be done when using Opac via Primo (OvP) as this also removes the holdings information for books. Turning the information off, also removes the Summary Holdings statement).

Question 8 - You need to find general journal articles on the subject of customer service training – how would you carry out this search? (You will need to sign in with your Coventry University username and password before starting the search)

This question was included to see how easy students would find the Articles tab to locate and use. At the time of testing the Articles tab had not been set up quite in the way that was needed and so inevitably there was quite a lot of prompting to ensure testers came up with a list of results that would be useful for the follow-up question (#9). Only half of the testers found the Articles tab without prompting. Of the 5 remaining testers, 2 really struggled to find somewhere to search for articles, and 1 tester attempted to use the Journal and Article options within the Books tab advanced search. All testers found the sign-in link straight away which was very encouraging. All testers needed prompting to use the General Quickset, however some did look in this drop-down menu before prompting which indicates that if there had been subject-based quicksets there, it may have been more intuitive. 2 testers encountered a blank screen after having signed in after trying to conduct their search - behind this Primo was actually conducting the search, however there was no indication of this and caused some confusion.
Recommendations:
• Provide instructions on the front page of Primo to indicate what students would be able to find in the Articles tab.
• For Ex Libris: ensure that a blank screen is not displayed when a search is conducted when sign in happens after search terms are entered – Ex Libris have been looking in to this, but it has yet to be fixed.
• Insert text into Quickset box asking students to select a subject area – this has been done.
• More usability testing once the Articles tab is set up with our quicksets (dependent on Primo Central installation).

Question 9 - Please try to view one of the articles you have found.

This question was to see how testers would attempt to get to the full text of the article. Ex Libris had named the tab which linked to our SFX installation as ‘Services’ and we wanted to see if this was an intuitive place for the testers to click. The ‘Services’ tab was located in the same place that the ‘View online’ tab was from the Books etc search tab. Each article on the results list came up as ‘No Full Text’ even if we had access – this is related to the information coming from SFX and was incorrect due to not having implemented the most recent version of SFX. The ‘No Full Text’ information in the availability line misled many of the testers, and we were required to ask them to ignore it and try and access the articles anyway. 8 of the testers were able to find the ‘Services’ link, however many commented that this wasn’t very clear, or that they would not expect to gain access through that tab. Only 4 of the testers clicked on ‘Services’ without prompting, and 2 testers became frustrated and did not complete the question. 1 tester accessed an article through an unexpected route – the links box in the full details screen. This was something that hadn’t been considered when writing the questions for the test. In further testing by library staff, it seems that the links box does not appear for all articles and as such cannot be considered a reliable link to full-text access.
Recommendations:
• Rename the ‘Services’ tab to something more intuitive – one tester suggested ‘check for full-text’ or ‘availability’. Another tester suggested using ‘Find it @ CU’ – we have re-named ‘Services’ as Find it @ CU – our SFX branding.
• Ensure that the availability line information is accurate – if this is not possible then turn it off completely to avoid misleading users – currently this is turned off and will be investigated again in the future.
• Do further testing on the links boxes.

Primo Usability Testing - Details Post #1

To try to keep this detailed write-up of our usability testing more manageable we've broken it down into 3 posts. This is the first one.

Introduction / Background

The initial implementation of Primo took place in mid-July 2010, and the usability testing took place within 2-3 weeks of this implementation after staff had attended a two-day Ex Libris training session. The aim of the testing was to see how intuitive the Primo interface was, and to gauge how useful students thought some of the additional functionality would be e.g. Facets, merging of records and FRBRisation, Reviews & Tags.

In terms of recruiting for the testing we knew that it would be difficult to get students in the middle of the summer vacation, however we used the details that we had taken during the name survey earlier in the year. This did give us all of our student testers, and we recruited 2 members of academic staff by asking Subject Librarians to contact lecturers they knew were helpful. We conducted 10 tests in total, with 7 of the testers coming from the Business, Environment & Society faculty, and 3 coming from Engineering & Computing. We were unable to recruit students from the Health & Life Sciences faculty, possibly due to course structures and placements. All testers had used the library catalogue and/or the eLibrary (MetaLib) before, but had mixed library search skills.

• 2 x academic staff
• 5 x undergraduate students (two 2nd years and three 3rd years)
• 3 x postgraduate students

We conducted the usability tests in a usability lab within the university. The lab allowed us to record audio and video of the participants, along with a close-up screen capture which showed how the participants navigated around the screen. Within the analysis below, only the audio and screen-captures were used.

To become familiar with the usability lab environment, and to test the questions that we had written, we asked a member of library staff with some experience of enquiry work to run through the questions for us as a pilot study. This provided valuable feedback on the wording of some questions which we were able to amend before beginning the testing for real.

The tests themselves consisted of 12 scripted questions along with time for discussion at the end (including questioning about the location of facets and the participants' understanding of Reviews & Tags). Each test took around 45 minutes to complete, including discussion. We tried to use scenario-based questions so that students would feel that the tasks they were being asked to do would represent how they would approach the library search interface in 'real life'.

Analysis and Recommendations

Section 1 - Searching for books and e-books

Question 1 - You want to find out if the Library has the book “Operations management” by Nigel Slack from your reading list. Can you check if the Library holds this book?

All 10 testers were able to complete the task of recognising whether the library stocked a particular book, however 4 testers required prompting to ensure that they had seen that we held the specific book in question. Of these 4 testers, 2 picked the same titled book by a different author, 1 picked a differently titled book by the same author, and 1 became distracted by the ‘view 3 versions’ link of a different book. 5 of the 6 testers who completed the task without prompting were able to do so in less than 1 minute. It was noted that all testers recognised the search box immediately, and 1 tester commented about being in the Books etc tab already.

Question 2 - Can you tell us what floor it is on and what number it’s at in the Library?

In this question we wanted to see where the testers would find the location of the book from on the screen. 9 out of 10 testers were able to give verbal confirmation of the floor and number that the book was held at. The other tester provided confirmation of the floor, but not the classmark. The majority of testers found this information in the availability line in either the brief or full screen. 2 testers found this information in the locations tab. 2 testers needed prompting to give either the floor or the classmark, and the average time taken to find this information was around 19 seconds.

Question 3 - Can you tell us how many copies are out on loan?

Only 3 of the testers managed to complete the task of finding out how many copies were on loan without prompting. 4 testers were unable to complete the task without prompting – many clicked on all tabs except the Locations tab. Some testers commented that they would not look there as it would only tell you where the book was held and not how many we had. Many testers thought that these details would be in the Details or More tabs. There were 3 testers who were unable to complete the task. One of these was due to not recognising the ‘expand’ buttons in the locations tab when there was more than one sublibrary. 3 testers thought that the ‘availability line’ status was clickable and that this was all the information there was available.
Recommendations:
• Re-label the locations tab to something that is more intuitive e.g. one tester suggested ‘availability’ – the ‘locations’ tab has been re-named as ‘Availability’.
• Ensure that if there are multiple sub-libraries that they expand automatically so that all copies are displayed – Ex Libris responded that this is not possible at this time.
• For Ex Libris: possibility of making the availability line status as clickable, taking users into the ‘locations’ tab?

Question 4 - If all copies of this edition were out on loan, can you see if there are any other editions available to borrow and if so how many editions?

9 out of 10 testers completed the task by acknowledging the ‘View x versions’ link and providing confirmation of the number of editions available. Of these, 4 testers had already clicked on the ‘View x versions’ link as part of the previous task and had already noted that it took them to a list of editions of the book. This indicated that we may need to change the labelling of the link to ensure that students would know that it meant ‘editions’ of the book and not ‘copies’. The one tester who did not complete the task was unsure of what an ‘edition’ meant and did not notice the link in either full or brief details screens. During test 7, the ‘View x versions’ link disappeared from the full details screen and so the tester was prompted to return to the brief results. Other testers had seen this link when the system was working correctly.
Recommendations:
• Re-name ‘View x versions’ as ‘View x editions’ – this has been done.
• Ensure that the edition statement appears within the record – we have added this to the ‘Details’ tab.

Question 5 - You are at home and need to find some books and don’t have time to come into the Library. Can you find a list of e-books on the subject of human resource management?

5 of the 10 testers were able to find a list of e-books using the e-book facet on the results screen, 4 testers were able to complete the task with prompting and one tester did not complete the task, but then found the answer whilst completing task 6. Several testers used the ‘Show Online Resources’ link but did not notice that this included journals and e-books. Several testers tried to find an ‘e-books’ option before attempting their search, 3 testers used the advanced search and looked in the format option expecting there to be an ‘e-book’ choice. 3 testers thought that e-books would be in the e-shelf area and were confused by this. 7 of the testers saw the e-book facet – 4 without prompting.
Recommendations:
• Include brief instructions to users on the front page telling them that they can refine their search after retrieving a list of results.
• Re-name ‘e-shelf’ as this caused confusion as testers believed this was where they would find e-resources – this has been re-named as My Favourites.
• Explore further ways of distinguishing between journals and books in the ‘Show Online Resources’ results list

Question 6 - Please try to access one of these items

In this task we wanted to see whether users would be able to see how to open an e-book from the results list, and whether they would be happy with viewing an e-book in the ‘letterbox’ preview screen that is automatically opened when a user clicks on ‘View Online’. There were two potential ways of opening the e-book into a bigger screen – an icon on the right-hand side that would enlarge the e-book into a bigger Primo window (the same view as would happen if a user was in the full-details screen rather than the brief results list view). The second option was a link on the left-hand side that said ‘Open Source in a New Window’ (OSINW) which would open the e-book in a full non-Primo window. Firstly, all of the testers noticed the ‘View Online’ link when either in the brief or full details screens and understood this to mean opening the e-book. Almost all of the testers were unhappy about attempting to view the e-book in the small letterbox view, and many were also reluctant to view the e-book in the bigger Primo window. 5 testers from the brief view clicked on the icon to open the e-book in a bigger screen. Only one tester saw the OSINW link from the brief details screen. Only 4 testers in total saw the OSINW link in either the brief or full details screens. 4 testers attempted to use the internal options within the e-book itself to try and make the e-book bigger – several testers became quite frustrated at not being able to open the e-book in a non-Primo window.
Recommendations:
• Configure Primo to ensure that when clicking on ‘View Online’ the e-book would automatically open in a full-screen, non-Primo window (preferred option) – this is being looked in to by Ex Libris and a fix is due to be released in an upcoming Service Pack.
• If above not possible – swap the links from the OSINW and icon options so that the icon would open a non-Primo window and the OSINW link opened in a Primo window (or preferably both open in a non-Primo window).
• Possibly attempt to re-name OSINW text to something clearer