1. Locate interface displaying search results at Coventry University
2. Locate displaying integrated item availability information at Coventry University
3. Sample user reservation display at Coventry University
Tuesday, 23 November 2010
Final blog post - appendix
By popular demand, here are the requested screen dumps...
1. Locate interface displaying search results at Coventry University
2. Locate displaying integrated item availability information at Coventry University
3. Sample user reservation display at Coventry University
1. Locate interface displaying search results at Coventry University
2. Locate displaying integrated item availability information at Coventry University
3. Sample user reservation display at Coventry University
Friday, 22 October 2010
COVPRIMO Final Blog Post
Background
Coventry University is a post-1992 University with around 14,000 FTEs. The student population of undergraduates and taught postgraduates is very diverse studying across a wide range of subject areas and located in five Schools and Faculties (Art and Design, Business Environment and Society, Engineering and Computing, Health and Life Sciences and Lifelong Learning). The emphasis is on teaching, with a strong commitment to e-learning complimenting face to face interaction in the classroom. Research activity is also important.
The Lanchester Library is a busy academic library, often with over 5,000 people passing through the gates each day. Around 350,000 items are stocked and over 500,000 issues / renewals are made per annum. Around 12,000 new items are purchased annually. Over a million full-text journal articles are downloaded each year and there are over 800,000 e-book accesses.
Intended outcome(s)
The main objective of the project was to implement Ex Libris’s Primo discovery tool as a single user interface for Coventry University’s users to make the starting point for all their Library searches. This should help to provide a more consistent user experience than searching the Library Catalogue, MetaLib (our federated search tool) and Equella (our institutional repository) individually. This should help to improve search efficiency and so increase the use of our e-resources.
The challenge
In a context in which the ‘information seeking behaviour’ was perceived to be changing and the expectation of intuitive and effective interfaces was emerging, we were conscious of the need to enhance our resource discovery environment.
It was felt that the Aleph Library Catalogue was outdated and had none of the common web 2.0 functionality that our users expected to see. We had developed it as much as we possibly could, introducing a pretty major overhaul 2 years ago. This was carried out locally by our Library Systems team following some usability testing and was a definite improvement. However there were many other changes that we wanted to introduce that just weren’t possible within the constraints of the Aleph OPAC set-up.
Another concern came from the fact that we were using MetaLib alongside the Aleph OPAC as the main route to access our e-resources. This meant that users had 2 different places to search for Library content, and sometimes lead to confusion over which interface they were searching and which one they should be using. Usability testing of MetaLib was carried out at the same time as the OPAC usability tests and revealed that it wasn’t as intuitive as we would have liked. Added to this we are also aware of the growing numbers of international students studying at Coventry. Our understanding from focus groups is that they tend to come to do postgraduate courses with no previous experience of e-resources and do not find MetaLib intuitive.
Authentication to the OPAC and to MetaLib is carried out using 2 different methods, meaning that users are required to login with different usernames and passwords. OPAC authentication is by barcode from the user’s University card, whereas MetaLib uses LDAP to authenticate with the person’s University username and password. We were also aware from responses to the questions in our ‘user satisfaction surveys’ relating to ease of access to online resources that this was an area which we could improve on.
As a result of this it was decided that a new interface that combined the functionality of both the OPAC and MetaLib, was authenticated using Shibboleth and that was very configurable would be a good investment.
Established practice
The established practice was using the Aleph OPAC to search for books and journals but using MetaLib to search for databases by name or by subject. MetaLib also has a Find e-Journal option. This displays holdings information about e-journals which have been activated in the SFX link resolver. If an e-journal is available and displays in this list a link will also be provided back to the OPAC if the journal is available in print. However, if a journal is only available in print this will not display in the Find e-Journal list within MetaLib. This will only be discoverable from the OPAC, although SFX will provide links to print only journals if an article from one of these titles is found in a database.
The LMS advantage
The introduction of Primo as a replacement for the OPAC and MetaLib should simplify any confusion over which interface users should be searching. It is also designed in a much more intuitive way and works like many other popular websites. During our usability testing favourable comparisons were made to Tesco Online and Amazon. Primo is also much more configurable than the Aleph OPAC and than MetaLib. This enables us to customise it much more easily, something that we have already done based on the usability testing that we carried out as part of this project.
The usability tests that we ran as part of the project were extremely helpful. They enabled us to identify aspects of the out-of-the-box Primo which we wanted to change, before the launch. The testing was also of interest to Ex Libris and we had a conference call with the Primo Product Manager in September. This enabled us to feed back the main issues which we believe our usability tests revealed. This included things which we weren’t currently able to change and Ex Libris are going to consider introducing these as future updates to the Primo product.
We are also introducing Primo Central, which provides pre-harvested and indexed metadata from a range of databases. This enables the user to quickly retrieve journal articles alongside their other content (books, journals, videos, DVDs, databases, etc.). Previously to search for journal articles a federated search would need to be done, which carries out a live search of selected databases, so tends to be slow. Primo Central avoids this problem as the metadata is already harvested. We are just in the process of introducing Primo Central, so hope to carry out more analysis of its functionality over the next few months.
Another benefit is that Primo is being configured to authenticate via Shibboleth. This means that there will be no confusion about what credentials users need to login. It will also have an additional benefit for users who login to Primo and then link to a Shibboleth-authenticated database. These users will not need to login to the database during the same session, as they will have already been authenticated.
Key points for effective practice
Key points to note for other practitioners are as follows:
The timescale we were working to for implementing Primo was rather ambitious. We were trying to introduce it before the end of August, which is what would have been required to allow Subject Librarians to prepare their teaching materials for the new term. This might have been possible if we hadn’t wanted to carry out the usability testing and make significant changes to the Primo front end. However, as our Primo installation was delayed until July, with our first training day taking place on July 22nd, we weren’t able to carry out the testing until the final week of July. We did make excellent progress in customising the front end throughout August, thanks to the hard work of our Ex Libris UK contact who helped us make numerous changes until his deserved vacation in the 2nd half of August. Unfortunately, while he was away there wasn’t much additional support available meaning progress was hindered.
A number of other major concerns meant that the launch of Primo had to be postponed anyway. The main one of these was the setting up of Shibboleth authentication, which required involvement from the Library Systems Team, our IT Services department and one of Ex Libris’s German offices. This work began in August and continued throughout September, and there was no possibility of going live until this had been introduced. The delay was made worse by the fact that IT Services were migrating from Shibboleth 1.3 to 2 in early September
The other really big change which we were waiting for was the introduction of Primo Central. This was being rolled out to Primo customers over the summer, but wasn’t available to us until October. Given that adding this was a major change in the way Primo functioned it seemed sensible to delay the launch until after this had been added and configured to our specifications.
Other problems encountered during the project were the technical challenges of the transfer of data from Aleph and Equella (the institutional repository). It was also difficult to obtain advice on suitable backup routines from Ex Libris.
Another issue was experienced around the installation of Syndetics ICE for indexed content enrichment. We were told by Ex Libris that Primo was compatible with ICE and we were keen to test this functionality. However, when we tried to arrange this we found out that Primo wasn’t currently working with ICE. We were told it wasn’t a major problem, but we’re still waiting for this to be set up.
From a communications point of view it wasn't always easy discussing involved / complex technical issues over the phone with our Ex Libris Project Manager who was based in Germany. It was also not clear what we could expect to be carried out by Ex Libris as part of their basic installation service. We did identify a long list of changes to the interface following our usability testing, and our Ex Libris UK contact was extremely helpful in working with us on these. However, with hindsight more detailed clarification over where responsibilities lay would have been helpful.
Conclusions and recommendations
We are happy with our Primo installation so far and keen to make it available to our users. It’s a shame that we weren’t able to release it before the start of the new academic year, but it was definitely the right decision to delay things until Shibboleth had been configured and Primo Central had been added. We will then be able to launch a beta version of Primo. We plan to display a feedback form on the site to request more input from our users. We also hope to carry out further usability testing on the customised Coventry version of Primo and on Primo Central. We will then be able to make further changes (if necessary) before launching Primo as our single user interface.
The feedback we had from our usability testers was very positive and we believe this will also be the case with our other users. Based on our experience of Primo so far we believe it will be a major improvement over the previous search experience at Coventry, particularly with the introduction of Primo Central.
Recommendations:
• Make sure you have the right mix of people on the project group to ensure that the teams most involved in using the new system can present an equal viewpoint and to ensure project ownership across different sections of staff.
• Under no circumstance underestimate the time & resource such a project requires.
• Involve external suppliers (in this case of our repository, Equella) at an early stage.
• Allow enough time to customise the front end. If you plan to make significant changes then it would be sensible to allow a minimum of 2 months, more if you are planning to carry out usability tests and need to allow time to analyse these.
• If you know you are going to be doing testing over the summer period when there aren’t many students around try to recruit testers who live locally before the end of the summer term.
• Carry out a pilot study before you do your real testing. This can help to reveal any questions that don’t work as intended and will enable you to make changes before doing the test with your volunteers.
• Make sure you involve Academic staff in your testing.
• Try to get access to a usability lab to carry out the tests and record what takes place for accuracy.
• Get feedback and buy-in from Subject Librarians and staff who work on enquiry desks as these will be the people who will be teaching students how to use the system
• Try to ensure you have accurate creation and maintenance of the Name Authority Files in your LMS. We didn’t have this in Aleph and it would have made FRBRisation much more accurate. This is something that we are now having to review so that we get one FRBR group for something instead of 2 or more.
Additional information
Project Blog: http://covprimo.blogspot.com/
Coventry University Primo installation: http://primo.coventry.ac.uk/primo_library/libweb/action/search.do?dscnt=1&fromLogin=true&dstmp=1279105993031&vid=COV_VU1&fromLogin=true
Ex Libris Primo Overview: http://www.exlibrisgroup.com/category/PrimoOverview
Coventry University is a post-1992 University with around 14,000 FTEs. The student population of undergraduates and taught postgraduates is very diverse studying across a wide range of subject areas and located in five Schools and Faculties (Art and Design, Business Environment and Society, Engineering and Computing, Health and Life Sciences and Lifelong Learning). The emphasis is on teaching, with a strong commitment to e-learning complimenting face to face interaction in the classroom. Research activity is also important.
The Lanchester Library is a busy academic library, often with over 5,000 people passing through the gates each day. Around 350,000 items are stocked and over 500,000 issues / renewals are made per annum. Around 12,000 new items are purchased annually. Over a million full-text journal articles are downloaded each year and there are over 800,000 e-book accesses.
Intended outcome(s)
The main objective of the project was to implement Ex Libris’s Primo discovery tool as a single user interface for Coventry University’s users to make the starting point for all their Library searches. This should help to provide a more consistent user experience than searching the Library Catalogue, MetaLib (our federated search tool) and Equella (our institutional repository) individually. This should help to improve search efficiency and so increase the use of our e-resources.
The challenge
In a context in which the ‘information seeking behaviour’ was perceived to be changing and the expectation of intuitive and effective interfaces was emerging, we were conscious of the need to enhance our resource discovery environment.
It was felt that the Aleph Library Catalogue was outdated and had none of the common web 2.0 functionality that our users expected to see. We had developed it as much as we possibly could, introducing a pretty major overhaul 2 years ago. This was carried out locally by our Library Systems team following some usability testing and was a definite improvement. However there were many other changes that we wanted to introduce that just weren’t possible within the constraints of the Aleph OPAC set-up.
Another concern came from the fact that we were using MetaLib alongside the Aleph OPAC as the main route to access our e-resources. This meant that users had 2 different places to search for Library content, and sometimes lead to confusion over which interface they were searching and which one they should be using. Usability testing of MetaLib was carried out at the same time as the OPAC usability tests and revealed that it wasn’t as intuitive as we would have liked. Added to this we are also aware of the growing numbers of international students studying at Coventry. Our understanding from focus groups is that they tend to come to do postgraduate courses with no previous experience of e-resources and do not find MetaLib intuitive.
Authentication to the OPAC and to MetaLib is carried out using 2 different methods, meaning that users are required to login with different usernames and passwords. OPAC authentication is by barcode from the user’s University card, whereas MetaLib uses LDAP to authenticate with the person’s University username and password. We were also aware from responses to the questions in our ‘user satisfaction surveys’ relating to ease of access to online resources that this was an area which we could improve on.
As a result of this it was decided that a new interface that combined the functionality of both the OPAC and MetaLib, was authenticated using Shibboleth and that was very configurable would be a good investment.
Established practice
The established practice was using the Aleph OPAC to search for books and journals but using MetaLib to search for databases by name or by subject. MetaLib also has a Find e-Journal option. This displays holdings information about e-journals which have been activated in the SFX link resolver. If an e-journal is available and displays in this list a link will also be provided back to the OPAC if the journal is available in print. However, if a journal is only available in print this will not display in the Find e-Journal list within MetaLib. This will only be discoverable from the OPAC, although SFX will provide links to print only journals if an article from one of these titles is found in a database.
The LMS advantage
The introduction of Primo as a replacement for the OPAC and MetaLib should simplify any confusion over which interface users should be searching. It is also designed in a much more intuitive way and works like many other popular websites. During our usability testing favourable comparisons were made to Tesco Online and Amazon. Primo is also much more configurable than the Aleph OPAC and than MetaLib. This enables us to customise it much more easily, something that we have already done based on the usability testing that we carried out as part of this project.
The usability tests that we ran as part of the project were extremely helpful. They enabled us to identify aspects of the out-of-the-box Primo which we wanted to change, before the launch. The testing was also of interest to Ex Libris and we had a conference call with the Primo Product Manager in September. This enabled us to feed back the main issues which we believe our usability tests revealed. This included things which we weren’t currently able to change and Ex Libris are going to consider introducing these as future updates to the Primo product.
We are also introducing Primo Central, which provides pre-harvested and indexed metadata from a range of databases. This enables the user to quickly retrieve journal articles alongside their other content (books, journals, videos, DVDs, databases, etc.). Previously to search for journal articles a federated search would need to be done, which carries out a live search of selected databases, so tends to be slow. Primo Central avoids this problem as the metadata is already harvested. We are just in the process of introducing Primo Central, so hope to carry out more analysis of its functionality over the next few months.
Another benefit is that Primo is being configured to authenticate via Shibboleth. This means that there will be no confusion about what credentials users need to login. It will also have an additional benefit for users who login to Primo and then link to a Shibboleth-authenticated database. These users will not need to login to the database during the same session, as they will have already been authenticated.
Key points for effective practice
Key points to note for other practitioners are as follows:
The timescale we were working to for implementing Primo was rather ambitious. We were trying to introduce it before the end of August, which is what would have been required to allow Subject Librarians to prepare their teaching materials for the new term. This might have been possible if we hadn’t wanted to carry out the usability testing and make significant changes to the Primo front end. However, as our Primo installation was delayed until July, with our first training day taking place on July 22nd, we weren’t able to carry out the testing until the final week of July. We did make excellent progress in customising the front end throughout August, thanks to the hard work of our Ex Libris UK contact who helped us make numerous changes until his deserved vacation in the 2nd half of August. Unfortunately, while he was away there wasn’t much additional support available meaning progress was hindered.
A number of other major concerns meant that the launch of Primo had to be postponed anyway. The main one of these was the setting up of Shibboleth authentication, which required involvement from the Library Systems Team, our IT Services department and one of Ex Libris’s German offices. This work began in August and continued throughout September, and there was no possibility of going live until this had been introduced. The delay was made worse by the fact that IT Services were migrating from Shibboleth 1.3 to 2 in early September
The other really big change which we were waiting for was the introduction of Primo Central. This was being rolled out to Primo customers over the summer, but wasn’t available to us until October. Given that adding this was a major change in the way Primo functioned it seemed sensible to delay the launch until after this had been added and configured to our specifications.
Other problems encountered during the project were the technical challenges of the transfer of data from Aleph and Equella (the institutional repository). It was also difficult to obtain advice on suitable backup routines from Ex Libris.
Another issue was experienced around the installation of Syndetics ICE for indexed content enrichment. We were told by Ex Libris that Primo was compatible with ICE and we were keen to test this functionality. However, when we tried to arrange this we found out that Primo wasn’t currently working with ICE. We were told it wasn’t a major problem, but we’re still waiting for this to be set up.
From a communications point of view it wasn't always easy discussing involved / complex technical issues over the phone with our Ex Libris Project Manager who was based in Germany. It was also not clear what we could expect to be carried out by Ex Libris as part of their basic installation service. We did identify a long list of changes to the interface following our usability testing, and our Ex Libris UK contact was extremely helpful in working with us on these. However, with hindsight more detailed clarification over where responsibilities lay would have been helpful.
Conclusions and recommendations
We are happy with our Primo installation so far and keen to make it available to our users. It’s a shame that we weren’t able to release it before the start of the new academic year, but it was definitely the right decision to delay things until Shibboleth had been configured and Primo Central had been added. We will then be able to launch a beta version of Primo. We plan to display a feedback form on the site to request more input from our users. We also hope to carry out further usability testing on the customised Coventry version of Primo and on Primo Central. We will then be able to make further changes (if necessary) before launching Primo as our single user interface.
The feedback we had from our usability testers was very positive and we believe this will also be the case with our other users. Based on our experience of Primo so far we believe it will be a major improvement over the previous search experience at Coventry, particularly with the introduction of Primo Central.
Recommendations:
• Make sure you have the right mix of people on the project group to ensure that the teams most involved in using the new system can present an equal viewpoint and to ensure project ownership across different sections of staff.
• Under no circumstance underestimate the time & resource such a project requires.
• Involve external suppliers (in this case of our repository, Equella) at an early stage.
• Allow enough time to customise the front end. If you plan to make significant changes then it would be sensible to allow a minimum of 2 months, more if you are planning to carry out usability tests and need to allow time to analyse these.
• If you know you are going to be doing testing over the summer period when there aren’t many students around try to recruit testers who live locally before the end of the summer term.
• Carry out a pilot study before you do your real testing. This can help to reveal any questions that don’t work as intended and will enable you to make changes before doing the test with your volunteers.
• Make sure you involve Academic staff in your testing.
• Try to get access to a usability lab to carry out the tests and record what takes place for accuracy.
• Get feedback and buy-in from Subject Librarians and staff who work on enquiry desks as these will be the people who will be teaching students how to use the system
• Try to ensure you have accurate creation and maintenance of the Name Authority Files in your LMS. We didn’t have this in Aleph and it would have made FRBRisation much more accurate. This is something that we are now having to review so that we get one FRBR group for something instead of 2 or more.
Additional information
Project Blog: http://covprimo.blogspot.com/
Coventry University Primo installation: http://primo.coventry.ac.uk/primo_library/libweb/action/search.do?dscnt=1&fromLogin=true&dstmp=1279105993031&vid=COV_VU1&fromLogin=true
Ex Libris Primo Overview: http://www.exlibrisgroup.com/category/PrimoOverview
Primo Usability Testing - Details Post #3
This is the third of our 3 posts giving a detailed write-up of our usability testing.
Section 3 – Finding Databases and Post-Test Discussion
Question 10 - Your lecturer has told you to search the database ASSIA (Applied Social Sciences Index & Abstracts). How would you find it and access it.
In this question we were attempting to find out the visibility of the Find Databases link (as it is only shown in the Articles etc. tab), and to see how testers used the functionality within the Lightbox to access a specific database. We were aware before the testing that an alternative method of finding a named database existed i.e. searching by name in the Books etc. tab and only 1 tester used this method. All 9 other testers used the Find Databases link – 3 testers needed prompting to either go in to the articles tab, or to look around the screen to see if they could see a link. 3 testers saw the Find Databases link straight away, and 3 others saw it after moving between the Books and Articles tabs. Some testers tried to use the options within the Advanced Search in the Articles tab before noticing the Find Databases link.
In terms of actually accessing ASSIA, of the 9 testers who used the Lightbox, 5 ticked on the checkbox expecting that it would open up the database. 2 of these then went on to click on ‘Selected Databases’ in the Lightbox which automatically took them out of the Lightbox and back to the Article etc tab search screen with the selected database as a ‘quickset’. This was very confusing for the testers as there was no warning or indication of what had happened. To access ASSIA the testers needed to click on the title of the database and 4 did this straight away without ticking the checkbox, and 3 others clicked on the title after ticking the checkbox and having nothing happen.
Recommendations:
• Make the Find Databases link appear in both the Articles etc tab and the Books etc tab – Ex Libris are looking in to this.
• Make the Find Databases link more visible – some testers commented that it was hidden away in the corner – we have aligned this to the left-hand side, immediately above the main search box and used a larger font.
• Provide instructions for students on finding named databases on the front page of Primo.
• Remove the checkboxes from the Lightbox and remove un-needed functionality (e.g. ‘Selected Databases’) – Ex Libris have removed the checkboxes and have taken out some of the functionality within the Lightbox. This needs more development work in the future.
Question 11 - ASSIA isn’t finding you the articles you want and you’re not sure what other databases you should use to find articles from the Business & Economics subject area. How would you find a list of databases recommended for this subject?
In this question we were looking at whether the testers would be able to find a list of databases relevant to a specific subject area from the information available to them in the Lightbox. Many of the testers were already in the Lightbox when they attempted this question and found a list of databases relatively quickly although many clicked on several drop-down menus before realising the information they wanted was in Category – this indicates that more work needed to be done to ensure that this was more intuitive. Some testers suggested using ‘Subject’ instead of ‘Category’ to indicate more clearly. Some testers also seemed to expect the list at the bottom of the Lightbox to automatically switch to the chosen subject area and it took a few seconds for them to click on the Find Databases link instead. This expectation could be due to their experiences within Metalib where the list automatically opens.
Recommendations:
• Re-name ‘category’ as something more intuitive – this has been re-named as ‘subject’.
• Insert text in to ‘Category’ box to indicate to students that they can choose a subject area i.e. ‘Select subject area’.
Post-Test Discussion / General Observations
Many testers thought that once they were used to the new site they would find it easy to use, especially as it had everything in one place unlike the library catalogue and eLibrary. Some suggested that the new site was more professional than the other library search tools they had used.
Facets – The majority of testers thought that the facets were best placed on the left-hand side of the screen – some testers mentioned websites such as Google and Tesco in comparison. All testers thought that the facets would be useful to them when searching library resources; however some commented that work could be done to make them more visible e.g. some testers didn’t notice them until they were pointed out. It was encouraging throughout the tests to see some testers using them on searches that didn’t explicitly require them to do so.
Reviews & Tags – None of the testers knew what a Tag was. When we explained to them that they could add their own keywords to individual catalogue records and asked if they thought this was useful some of them did think it would be. However, most of them were thinking of using the Tags as a way for them to retrieve records at a later date. It would probably be easier for them to just save these records to their personal Primo space (e-Shelf or My Favourites as we renamed it). When a scenario of tagging records that could be useful for other students working on a joint project was suggested most testers thought that this was a good idea. However, the fact that most of them didn’t think of this or any other social-networking way of using tags seems to imply that this functionality may not be well used. We are planning to leave the Reviews & Tags function switched on and see how much use it gets.
Other Actions based on the Usability Tests:
• The ‘More’ tab was removed as it duplicated information found in other tabs and was felt to be irrelevant.
• The main tabs were relabelled as ‘Books, Journals, Media & More’ and ‘Articles & More’ to differentiate the types of content in each tab.
• Some tabs were re-ordered to give more prominence to actual availability e.g. the ‘Locations’ tab (re-named ‘Availability’) is now either the first or second in the list rather than appearing after the ‘Request’ tab.
• The ‘New Search’ link needed to be made much clearer – a few testers commented that they couldn’t find a way to clear their search – we have left-justified this and made the font bigger.
• The e-shelf was misleading for quite a few testers, as they thought this was an area where they could search for e-resources – we have now re-named this as My Favourites in keeping with the general trends found on other mainstream websites.
Section 3 – Finding Databases and Post-Test Discussion
Question 10 - Your lecturer has told you to search the database ASSIA (Applied Social Sciences Index & Abstracts). How would you find it and access it.
In this question we were attempting to find out the visibility of the Find Databases link (as it is only shown in the Articles etc. tab), and to see how testers used the functionality within the Lightbox to access a specific database. We were aware before the testing that an alternative method of finding a named database existed i.e. searching by name in the Books etc. tab and only 1 tester used this method. All 9 other testers used the Find Databases link – 3 testers needed prompting to either go in to the articles tab, or to look around the screen to see if they could see a link. 3 testers saw the Find Databases link straight away, and 3 others saw it after moving between the Books and Articles tabs. Some testers tried to use the options within the Advanced Search in the Articles tab before noticing the Find Databases link.
In terms of actually accessing ASSIA, of the 9 testers who used the Lightbox, 5 ticked on the checkbox expecting that it would open up the database. 2 of these then went on to click on ‘Selected Databases’ in the Lightbox which automatically took them out of the Lightbox and back to the Article etc tab search screen with the selected database as a ‘quickset’. This was very confusing for the testers as there was no warning or indication of what had happened. To access ASSIA the testers needed to click on the title of the database and 4 did this straight away without ticking the checkbox, and 3 others clicked on the title after ticking the checkbox and having nothing happen.
Recommendations:
• Make the Find Databases link appear in both the Articles etc tab and the Books etc tab – Ex Libris are looking in to this.
• Make the Find Databases link more visible – some testers commented that it was hidden away in the corner – we have aligned this to the left-hand side, immediately above the main search box and used a larger font.
• Provide instructions for students on finding named databases on the front page of Primo.
• Remove the checkboxes from the Lightbox and remove un-needed functionality (e.g. ‘Selected Databases’) – Ex Libris have removed the checkboxes and have taken out some of the functionality within the Lightbox. This needs more development work in the future.
Question 11 - ASSIA isn’t finding you the articles you want and you’re not sure what other databases you should use to find articles from the Business & Economics subject area. How would you find a list of databases recommended for this subject?
In this question we were looking at whether the testers would be able to find a list of databases relevant to a specific subject area from the information available to them in the Lightbox. Many of the testers were already in the Lightbox when they attempted this question and found a list of databases relatively quickly although many clicked on several drop-down menus before realising the information they wanted was in Category – this indicates that more work needed to be done to ensure that this was more intuitive. Some testers suggested using ‘Subject’ instead of ‘Category’ to indicate more clearly. Some testers also seemed to expect the list at the bottom of the Lightbox to automatically switch to the chosen subject area and it took a few seconds for them to click on the Find Databases link instead. This expectation could be due to their experiences within Metalib where the list automatically opens.
Recommendations:
• Re-name ‘category’ as something more intuitive – this has been re-named as ‘subject’.
• Insert text in to ‘Category’ box to indicate to students that they can choose a subject area i.e. ‘Select subject area’.
Post-Test Discussion / General Observations
Many testers thought that once they were used to the new site they would find it easy to use, especially as it had everything in one place unlike the library catalogue and eLibrary. Some suggested that the new site was more professional than the other library search tools they had used.
Facets – The majority of testers thought that the facets were best placed on the left-hand side of the screen – some testers mentioned websites such as Google and Tesco in comparison. All testers thought that the facets would be useful to them when searching library resources; however some commented that work could be done to make them more visible e.g. some testers didn’t notice them until they were pointed out. It was encouraging throughout the tests to see some testers using them on searches that didn’t explicitly require them to do so.
Reviews & Tags – None of the testers knew what a Tag was. When we explained to them that they could add their own keywords to individual catalogue records and asked if they thought this was useful some of them did think it would be. However, most of them were thinking of using the Tags as a way for them to retrieve records at a later date. It would probably be easier for them to just save these records to their personal Primo space (e-Shelf or My Favourites as we renamed it). When a scenario of tagging records that could be useful for other students working on a joint project was suggested most testers thought that this was a good idea. However, the fact that most of them didn’t think of this or any other social-networking way of using tags seems to imply that this functionality may not be well used. We are planning to leave the Reviews & Tags function switched on and see how much use it gets.
Other Actions based on the Usability Tests:
• The ‘More’ tab was removed as it duplicated information found in other tabs and was felt to be irrelevant.
• The main tabs were relabelled as ‘Books, Journals, Media & More’ and ‘Articles & More’ to differentiate the types of content in each tab.
• Some tabs were re-ordered to give more prominence to actual availability e.g. the ‘Locations’ tab (re-named ‘Availability’) is now either the first or second in the list rather than appearing after the ‘Request’ tab.
• The ‘New Search’ link needed to be made much clearer – a few testers commented that they couldn’t find a way to clear their search – we have left-justified this and made the font bigger.
• The e-shelf was misleading for quite a few testers, as they thought this was an area where they could search for e-resources – we have now re-named this as My Favourites in keeping with the general trends found on other mainstream websites.
Final technical observations
For those interested in a quick summary of the technical aspects of this project, the implementation notes which form part of the 'one minute blog' post may prove useful.
Here I thought I'd concentrate on a couple of technical areas that have eaten away at the time.
Shibboleth implementation is one of the main culprits in this category. Although this formed part of the contract with our supplier, there were still significant obligations, both for library systems staff as well as IT Services staff. The university already had a Shibboleth idp service in place but on the library side we started from scratch and encountered a fairly steep learning curve. There's a whole new vocabulary to learn when addressing issues that surround authentication - it's wasn't immediately obvious where to go for help. Fairly early on our PM, sporting a large grin, pronounced, "I don't do Shibboleth!" I can see why. Added to this, the upgrade to the unversity's idp from Shibboleth version 1.3 to version 2.0, right in the middle of our testing phase, set us back considerably.
Following on from the Shibboleth issue, we were also aware we needed to upgrade the OPAC authentication functionality which still made use of our 14 digit patron barcodes. In order to move to Shibboleth authentication, it would prove necessary to insert patron usernames into their Aleph records. Inserting this data into the relevant field (z308_rec_key) with the appropriate flag would prove time-consuming. Occasionally the unversity issues a new username to a user - dealing with updates of this type continues to create some problems.
Other time-consuming activities included establishing data export / import routines for the source systems. The library management system Aleph proved troublesome - plenty of room for improvement in the supplier's documentation here. Having said that, the work our PM carried out on the university's repository product, Equella, proved invaluable.
Finally I'm sure my colleague who worked through many of our technical troubles (and is currently taking a well-earned holiday) would want me to mention the unavoidable overhead that is incurred when installing separate third party products which all go to form part of the complete installation.
By definition any summary such as this will struggle to give a complete picture; I hope these observations have at least provided a useful snapshot.
Here I thought I'd concentrate on a couple of technical areas that have eaten away at the time.
Shibboleth implementation is one of the main culprits in this category. Although this formed part of the contract with our supplier, there were still significant obligations, both for library systems staff as well as IT Services staff. The university already had a Shibboleth idp service in place but on the library side we started from scratch and encountered a fairly steep learning curve. There's a whole new vocabulary to learn when addressing issues that surround authentication - it's wasn't immediately obvious where to go for help. Fairly early on our PM, sporting a large grin, pronounced, "I don't do Shibboleth!" I can see why. Added to this, the upgrade to the unversity's idp from Shibboleth version 1.3 to version 2.0, right in the middle of our testing phase, set us back considerably.
Following on from the Shibboleth issue, we were also aware we needed to upgrade the OPAC authentication functionality which still made use of our 14 digit patron barcodes. In order to move to Shibboleth authentication, it would prove necessary to insert patron usernames into their Aleph records. Inserting this data into the relevant field (z308_rec_key) with the appropriate flag would prove time-consuming. Occasionally the unversity issues a new username to a user - dealing with updates of this type continues to create some problems.
Other time-consuming activities included establishing data export / import routines for the source systems. The library management system Aleph proved troublesome - plenty of room for improvement in the supplier's documentation here. Having said that, the work our PM carried out on the university's repository product, Equella, proved invaluable.
Finally I'm sure my colleague who worked through many of our technical troubles (and is currently taking a well-earned holiday) would want me to mention the unavoidable overhead that is incurred when installing separate third party products which all go to form part of the complete installation.
By definition any summary such as this will struggle to give a complete picture; I hope these observations have at least provided a useful snapshot.
Thursday, 21 October 2010
Primo Usability Testing - Details Post #2
This is the second of our 3 posts giving a detailed write-up of our usability testing.
Section 2 - Searching for known journals and journal articles
Question 7 - Your lecturer has told you to read an article in the most recent issue of the journal, the Harvard Business Review. Check to see if this is available in print, electronically or in both formats.
In this task we wanted to see how the testers would search for an individual journal. There are clear options for this in the library catalogue and eLibrary; however Primo does not explicitly offer an option for this at first glance. Only 2 of the 10 testers were able to correctly identify that we had both print and electronic access to the journal. The majority of testers saw ‘Online access’ and ‘Media format: E-Journal’ and suggested that we only had electronic access to the journal. Only two testers clicked on the Locations tab which is where they would find details of the print access. One tester found the print access by clicking on the ‘More’ tab and noticing the ‘Check print availability’ option in the SFX menu – however they did not verbally confirm if there was electronic access. 4 of the testers tried to search for the journal in the ‘Articles etc’ tab as they assumed that this was where they would find journals. Some of these were prompted to look at the ‘etc’ part of the tab name – one tester stated that they thought this was very unprofessional terminology to use in a higher education environment. Two of the testers used the Advanced Search option in the Books etc tab and chose ‘Journal’ from the format. Other testers commented that they had previously used the Journal Search or Find e-journal options with the Library Catalogue and eLibrary. This indicates that the testers were looking for ‘somewhere’ to search for journals and wouldn’t necessarily use the Books etc tab for this purpose.
Recommendations:
• Re-name the ‘Books etc’ tab to include the word ‘journal’ and create a more professional-sounding name – this would indicate to users more clearly where they should search for different material types – this has been re-named ‘Books, Journals, Media & More’.
• Find a way of making print journal access more visible to the user – possibly re-naming the Locations tab could help with this.
• Check with Ex Libris to see if different text can be displayed in the ‘Availability Status’ line (would also help with e-books – where print availability is also hidden) – Ex Libris responded that a different text can be displayed – we are still looking in alternative wording.
• Change the information in the Media Format section of the ‘Details’ tab to include both print and electronic access – if this is not possible would removing it altogether be an option? – we have removed the Media format option from this tab.
Question 7a - What years are available and from where?
Out of 9 testers who attempted the question, only 2 were able to correctly identify the years of access as from 1952 (print) and from 1922 (electronically). The majority of testers thought that we only had access to one year of the journal – 1922. This date was mostly identified in the results list, the details tab or in the sfx menu. 1 tester did not verbally conform any dates at all, despite scrolling through a list of journal issues in EBSCO. Unfortunately, due to the lack of recognition of print availability (as shown in question 7) it was not possible to effectively evaluate the information shown in the journals locations tab, although the testers who found it were confused by the amount of information on the screen and took some time to notice the Summary Holdings statement showing the years available.
Recommendations:
• If possible include the journal ‘from and to dates’ in the brief display and full details tabs (not possible due to cataloguing implications)
• If the above is not possible, then remove the date entirely from these sections as it could be deemed to be very misleading – we have removed the date for journals.
• Try to remove all of the information in the journals locations tab to make the Summary Holdings statement stand out more prominently (unfortunately this is unable to be done when using Opac via Primo (OvP) as this also removes the holdings information for books. Turning the information off, also removes the Summary Holdings statement).
Question 8 - You need to find general journal articles on the subject of customer service training – how would you carry out this search? (You will need to sign in with your Coventry University username and password before starting the search)
This question was included to see how easy students would find the Articles tab to locate and use. At the time of testing the Articles tab had not been set up quite in the way that was needed and so inevitably there was quite a lot of prompting to ensure testers came up with a list of results that would be useful for the follow-up question (#9). Only half of the testers found the Articles tab without prompting. Of the 5 remaining testers, 2 really struggled to find somewhere to search for articles, and 1 tester attempted to use the Journal and Article options within the Books tab advanced search. All testers found the sign-in link straight away which was very encouraging. All testers needed prompting to use the General Quickset, however some did look in this drop-down menu before prompting which indicates that if there had been subject-based quicksets there, it may have been more intuitive. 2 testers encountered a blank screen after having signed in after trying to conduct their search - behind this Primo was actually conducting the search, however there was no indication of this and caused some confusion.
Recommendations:
• Provide instructions on the front page of Primo to indicate what students would be able to find in the Articles tab.
• For Ex Libris: ensure that a blank screen is not displayed when a search is conducted when sign in happens after search terms are entered – Ex Libris have been looking in to this, but it has yet to be fixed.
• Insert text into Quickset box asking students to select a subject area – this has been done.
• More usability testing once the Articles tab is set up with our quicksets (dependent on Primo Central installation).
Question 9 - Please try to view one of the articles you have found.
This question was to see how testers would attempt to get to the full text of the article. Ex Libris had named the tab which linked to our SFX installation as ‘Services’ and we wanted to see if this was an intuitive place for the testers to click. The ‘Services’ tab was located in the same place that the ‘View online’ tab was from the Books etc search tab. Each article on the results list came up as ‘No Full Text’ even if we had access – this is related to the information coming from SFX and was incorrect due to not having implemented the most recent version of SFX. The ‘No Full Text’ information in the availability line misled many of the testers, and we were required to ask them to ignore it and try and access the articles anyway. 8 of the testers were able to find the ‘Services’ link, however many commented that this wasn’t very clear, or that they would not expect to gain access through that tab. Only 4 of the testers clicked on ‘Services’ without prompting, and 2 testers became frustrated and did not complete the question. 1 tester accessed an article through an unexpected route – the links box in the full details screen. This was something that hadn’t been considered when writing the questions for the test. In further testing by library staff, it seems that the links box does not appear for all articles and as such cannot be considered a reliable link to full-text access.
Recommendations:
• Rename the ‘Services’ tab to something more intuitive – one tester suggested ‘check for full-text’ or ‘availability’. Another tester suggested using ‘Find it @ CU’ – we have re-named ‘Services’ as Find it @ CU – our SFX branding.
• Ensure that the availability line information is accurate – if this is not possible then turn it off completely to avoid misleading users – currently this is turned off and will be investigated again in the future.
• Do further testing on the links boxes.
Section 2 - Searching for known journals and journal articles
Question 7 - Your lecturer has told you to read an article in the most recent issue of the journal, the Harvard Business Review. Check to see if this is available in print, electronically or in both formats.
In this task we wanted to see how the testers would search for an individual journal. There are clear options for this in the library catalogue and eLibrary; however Primo does not explicitly offer an option for this at first glance. Only 2 of the 10 testers were able to correctly identify that we had both print and electronic access to the journal. The majority of testers saw ‘Online access’ and ‘Media format: E-Journal’ and suggested that we only had electronic access to the journal. Only two testers clicked on the Locations tab which is where they would find details of the print access. One tester found the print access by clicking on the ‘More’ tab and noticing the ‘Check print availability’ option in the SFX menu – however they did not verbally confirm if there was electronic access. 4 of the testers tried to search for the journal in the ‘Articles etc’ tab as they assumed that this was where they would find journals. Some of these were prompted to look at the ‘etc’ part of the tab name – one tester stated that they thought this was very unprofessional terminology to use in a higher education environment. Two of the testers used the Advanced Search option in the Books etc tab and chose ‘Journal’ from the format. Other testers commented that they had previously used the Journal Search or Find e-journal options with the Library Catalogue and eLibrary. This indicates that the testers were looking for ‘somewhere’ to search for journals and wouldn’t necessarily use the Books etc tab for this purpose.
Recommendations:
• Re-name the ‘Books etc’ tab to include the word ‘journal’ and create a more professional-sounding name – this would indicate to users more clearly where they should search for different material types – this has been re-named ‘Books, Journals, Media & More’.
• Find a way of making print journal access more visible to the user – possibly re-naming the Locations tab could help with this.
• Check with Ex Libris to see if different text can be displayed in the ‘Availability Status’ line (would also help with e-books – where print availability is also hidden) – Ex Libris responded that a different text can be displayed – we are still looking in alternative wording.
• Change the information in the Media Format section of the ‘Details’ tab to include both print and electronic access – if this is not possible would removing it altogether be an option? – we have removed the Media format option from this tab.
Question 7a - What years are available and from where?
Out of 9 testers who attempted the question, only 2 were able to correctly identify the years of access as from 1952 (print) and from 1922 (electronically). The majority of testers thought that we only had access to one year of the journal – 1922. This date was mostly identified in the results list, the details tab or in the sfx menu. 1 tester did not verbally conform any dates at all, despite scrolling through a list of journal issues in EBSCO. Unfortunately, due to the lack of recognition of print availability (as shown in question 7) it was not possible to effectively evaluate the information shown in the journals locations tab, although the testers who found it were confused by the amount of information on the screen and took some time to notice the Summary Holdings statement showing the years available.
Recommendations:
• If possible include the journal ‘from and to dates’ in the brief display and full details tabs (not possible due to cataloguing implications)
• If the above is not possible, then remove the date entirely from these sections as it could be deemed to be very misleading – we have removed the date for journals.
• Try to remove all of the information in the journals locations tab to make the Summary Holdings statement stand out more prominently (unfortunately this is unable to be done when using Opac via Primo (OvP) as this also removes the holdings information for books. Turning the information off, also removes the Summary Holdings statement).
Question 8 - You need to find general journal articles on the subject of customer service training – how would you carry out this search? (You will need to sign in with your Coventry University username and password before starting the search)
This question was included to see how easy students would find the Articles tab to locate and use. At the time of testing the Articles tab had not been set up quite in the way that was needed and so inevitably there was quite a lot of prompting to ensure testers came up with a list of results that would be useful for the follow-up question (#9). Only half of the testers found the Articles tab without prompting. Of the 5 remaining testers, 2 really struggled to find somewhere to search for articles, and 1 tester attempted to use the Journal and Article options within the Books tab advanced search. All testers found the sign-in link straight away which was very encouraging. All testers needed prompting to use the General Quickset, however some did look in this drop-down menu before prompting which indicates that if there had been subject-based quicksets there, it may have been more intuitive. 2 testers encountered a blank screen after having signed in after trying to conduct their search - behind this Primo was actually conducting the search, however there was no indication of this and caused some confusion.
Recommendations:
• Provide instructions on the front page of Primo to indicate what students would be able to find in the Articles tab.
• For Ex Libris: ensure that a blank screen is not displayed when a search is conducted when sign in happens after search terms are entered – Ex Libris have been looking in to this, but it has yet to be fixed.
• Insert text into Quickset box asking students to select a subject area – this has been done.
• More usability testing once the Articles tab is set up with our quicksets (dependent on Primo Central installation).
Question 9 - Please try to view one of the articles you have found.
This question was to see how testers would attempt to get to the full text of the article. Ex Libris had named the tab which linked to our SFX installation as ‘Services’ and we wanted to see if this was an intuitive place for the testers to click. The ‘Services’ tab was located in the same place that the ‘View online’ tab was from the Books etc search tab. Each article on the results list came up as ‘No Full Text’ even if we had access – this is related to the information coming from SFX and was incorrect due to not having implemented the most recent version of SFX. The ‘No Full Text’ information in the availability line misled many of the testers, and we were required to ask them to ignore it and try and access the articles anyway. 8 of the testers were able to find the ‘Services’ link, however many commented that this wasn’t very clear, or that they would not expect to gain access through that tab. Only 4 of the testers clicked on ‘Services’ without prompting, and 2 testers became frustrated and did not complete the question. 1 tester accessed an article through an unexpected route – the links box in the full details screen. This was something that hadn’t been considered when writing the questions for the test. In further testing by library staff, it seems that the links box does not appear for all articles and as such cannot be considered a reliable link to full-text access.
Recommendations:
• Rename the ‘Services’ tab to something more intuitive – one tester suggested ‘check for full-text’ or ‘availability’. Another tester suggested using ‘Find it @ CU’ – we have re-named ‘Services’ as Find it @ CU – our SFX branding.
• Ensure that the availability line information is accurate – if this is not possible then turn it off completely to avoid misleading users – currently this is turned off and will be investigated again in the future.
• Do further testing on the links boxes.
Primo Usability Testing - Details Post #1
To try to keep this detailed write-up of our usability testing more manageable we've broken it down into 3 posts. This is the first one.
Introduction / Background
The initial implementation of Primo took place in mid-July 2010, and the usability testing took place within 2-3 weeks of this implementation after staff had attended a two-day Ex Libris training session. The aim of the testing was to see how intuitive the Primo interface was, and to gauge how useful students thought some of the additional functionality would be e.g. Facets, merging of records and FRBRisation, Reviews & Tags.
In terms of recruiting for the testing we knew that it would be difficult to get students in the middle of the summer vacation, however we used the details that we had taken during the name survey earlier in the year. This did give us all of our student testers, and we recruited 2 members of academic staff by asking Subject Librarians to contact lecturers they knew were helpful. We conducted 10 tests in total, with 7 of the testers coming from the Business, Environment & Society faculty, and 3 coming from Engineering & Computing. We were unable to recruit students from the Health & Life Sciences faculty, possibly due to course structures and placements. All testers had used the library catalogue and/or the eLibrary (MetaLib) before, but had mixed library search skills.
• 2 x academic staff
• 5 x undergraduate students (two 2nd years and three 3rd years)
• 3 x postgraduate students
We conducted the usability tests in a usability lab within the university. The lab allowed us to record audio and video of the participants, along with a close-up screen capture which showed how the participants navigated around the screen. Within the analysis below, only the audio and screen-captures were used.
To become familiar with the usability lab environment, and to test the questions that we had written, we asked a member of library staff with some experience of enquiry work to run through the questions for us as a pilot study. This provided valuable feedback on the wording of some questions which we were able to amend before beginning the testing for real.
The tests themselves consisted of 12 scripted questions along with time for discussion at the end (including questioning about the location of facets and the participants' understanding of Reviews & Tags). Each test took around 45 minutes to complete, including discussion. We tried to use scenario-based questions so that students would feel that the tasks they were being asked to do would represent how they would approach the library search interface in 'real life'.
Analysis and Recommendations
Section 1 - Searching for books and e-books
Question 1 - You want to find out if the Library has the book “Operations management” by Nigel Slack from your reading list. Can you check if the Library holds this book?
All 10 testers were able to complete the task of recognising whether the library stocked a particular book, however 4 testers required prompting to ensure that they had seen that we held the specific book in question. Of these 4 testers, 2 picked the same titled book by a different author, 1 picked a differently titled book by the same author, and 1 became distracted by the ‘view 3 versions’ link of a different book. 5 of the 6 testers who completed the task without prompting were able to do so in less than 1 minute. It was noted that all testers recognised the search box immediately, and 1 tester commented about being in the Books etc tab already.
Question 2 - Can you tell us what floor it is on and what number it’s at in the Library?
In this question we wanted to see where the testers would find the location of the book from on the screen. 9 out of 10 testers were able to give verbal confirmation of the floor and number that the book was held at. The other tester provided confirmation of the floor, but not the classmark. The majority of testers found this information in the availability line in either the brief or full screen. 2 testers found this information in the locations tab. 2 testers needed prompting to give either the floor or the classmark, and the average time taken to find this information was around 19 seconds.
Question 3 - Can you tell us how many copies are out on loan?
Only 3 of the testers managed to complete the task of finding out how many copies were on loan without prompting. 4 testers were unable to complete the task without prompting – many clicked on all tabs except the Locations tab. Some testers commented that they would not look there as it would only tell you where the book was held and not how many we had. Many testers thought that these details would be in the Details or More tabs. There were 3 testers who were unable to complete the task. One of these was due to not recognising the ‘expand’ buttons in the locations tab when there was more than one sublibrary. 3 testers thought that the ‘availability line’ status was clickable and that this was all the information there was available.
Recommendations:
• Re-label the locations tab to something that is more intuitive e.g. one tester suggested ‘availability’ – the ‘locations’ tab has been re-named as ‘Availability’.
• Ensure that if there are multiple sub-libraries that they expand automatically so that all copies are displayed – Ex Libris responded that this is not possible at this time.
• For Ex Libris: possibility of making the availability line status as clickable, taking users into the ‘locations’ tab?
Question 4 - If all copies of this edition were out on loan, can you see if there are any other editions available to borrow and if so how many editions?
9 out of 10 testers completed the task by acknowledging the ‘View x versions’ link and providing confirmation of the number of editions available. Of these, 4 testers had already clicked on the ‘View x versions’ link as part of the previous task and had already noted that it took them to a list of editions of the book. This indicated that we may need to change the labelling of the link to ensure that students would know that it meant ‘editions’ of the book and not ‘copies’. The one tester who did not complete the task was unsure of what an ‘edition’ meant and did not notice the link in either full or brief details screens. During test 7, the ‘View x versions’ link disappeared from the full details screen and so the tester was prompted to return to the brief results. Other testers had seen this link when the system was working correctly.
Recommendations:
• Re-name ‘View x versions’ as ‘View x editions’ – this has been done.
• Ensure that the edition statement appears within the record – we have added this to the ‘Details’ tab.
Question 5 - You are at home and need to find some books and don’t have time to come into the Library. Can you find a list of e-books on the subject of human resource management?
5 of the 10 testers were able to find a list of e-books using the e-book facet on the results screen, 4 testers were able to complete the task with prompting and one tester did not complete the task, but then found the answer whilst completing task 6. Several testers used the ‘Show Online Resources’ link but did not notice that this included journals and e-books. Several testers tried to find an ‘e-books’ option before attempting their search, 3 testers used the advanced search and looked in the format option expecting there to be an ‘e-book’ choice. 3 testers thought that e-books would be in the e-shelf area and were confused by this. 7 of the testers saw the e-book facet – 4 without prompting.
Recommendations:
• Include brief instructions to users on the front page telling them that they can refine their search after retrieving a list of results.
• Re-name ‘e-shelf’ as this caused confusion as testers believed this was where they would find e-resources – this has been re-named as My Favourites.
• Explore further ways of distinguishing between journals and books in the ‘Show Online Resources’ results list
Question 6 - Please try to access one of these items
In this task we wanted to see whether users would be able to see how to open an e-book from the results list, and whether they would be happy with viewing an e-book in the ‘letterbox’ preview screen that is automatically opened when a user clicks on ‘View Online’. There were two potential ways of opening the e-book into a bigger screen – an icon on the right-hand side that would enlarge the e-book into a bigger Primo window (the same view as would happen if a user was in the full-details screen rather than the brief results list view). The second option was a link on the left-hand side that said ‘Open Source in a New Window’ (OSINW) which would open the e-book in a full non-Primo window. Firstly, all of the testers noticed the ‘View Online’ link when either in the brief or full details screens and understood this to mean opening the e-book. Almost all of the testers were unhappy about attempting to view the e-book in the small letterbox view, and many were also reluctant to view the e-book in the bigger Primo window. 5 testers from the brief view clicked on the icon to open the e-book in a bigger screen. Only one tester saw the OSINW link from the brief details screen. Only 4 testers in total saw the OSINW link in either the brief or full details screens. 4 testers attempted to use the internal options within the e-book itself to try and make the e-book bigger – several testers became quite frustrated at not being able to open the e-book in a non-Primo window.
Recommendations:
• Configure Primo to ensure that when clicking on ‘View Online’ the e-book would automatically open in a full-screen, non-Primo window (preferred option) – this is being looked in to by Ex Libris and a fix is due to be released in an upcoming Service Pack.
• If above not possible – swap the links from the OSINW and icon options so that the icon would open a non-Primo window and the OSINW link opened in a Primo window (or preferably both open in a non-Primo window).
• Possibly attempt to re-name OSINW text to something clearer
Introduction / Background
The initial implementation of Primo took place in mid-July 2010, and the usability testing took place within 2-3 weeks of this implementation after staff had attended a two-day Ex Libris training session. The aim of the testing was to see how intuitive the Primo interface was, and to gauge how useful students thought some of the additional functionality would be e.g. Facets, merging of records and FRBRisation, Reviews & Tags.
In terms of recruiting for the testing we knew that it would be difficult to get students in the middle of the summer vacation, however we used the details that we had taken during the name survey earlier in the year. This did give us all of our student testers, and we recruited 2 members of academic staff by asking Subject Librarians to contact lecturers they knew were helpful. We conducted 10 tests in total, with 7 of the testers coming from the Business, Environment & Society faculty, and 3 coming from Engineering & Computing. We were unable to recruit students from the Health & Life Sciences faculty, possibly due to course structures and placements. All testers had used the library catalogue and/or the eLibrary (MetaLib) before, but had mixed library search skills.
• 2 x academic staff
• 5 x undergraduate students (two 2nd years and three 3rd years)
• 3 x postgraduate students
We conducted the usability tests in a usability lab within the university. The lab allowed us to record audio and video of the participants, along with a close-up screen capture which showed how the participants navigated around the screen. Within the analysis below, only the audio and screen-captures were used.
To become familiar with the usability lab environment, and to test the questions that we had written, we asked a member of library staff with some experience of enquiry work to run through the questions for us as a pilot study. This provided valuable feedback on the wording of some questions which we were able to amend before beginning the testing for real.
The tests themselves consisted of 12 scripted questions along with time for discussion at the end (including questioning about the location of facets and the participants' understanding of Reviews & Tags). Each test took around 45 minutes to complete, including discussion. We tried to use scenario-based questions so that students would feel that the tasks they were being asked to do would represent how they would approach the library search interface in 'real life'.
Analysis and Recommendations
Section 1 - Searching for books and e-books
Question 1 - You want to find out if the Library has the book “Operations management” by Nigel Slack from your reading list. Can you check if the Library holds this book?
All 10 testers were able to complete the task of recognising whether the library stocked a particular book, however 4 testers required prompting to ensure that they had seen that we held the specific book in question. Of these 4 testers, 2 picked the same titled book by a different author, 1 picked a differently titled book by the same author, and 1 became distracted by the ‘view 3 versions’ link of a different book. 5 of the 6 testers who completed the task without prompting were able to do so in less than 1 minute. It was noted that all testers recognised the search box immediately, and 1 tester commented about being in the Books etc tab already.
Question 2 - Can you tell us what floor it is on and what number it’s at in the Library?
In this question we wanted to see where the testers would find the location of the book from on the screen. 9 out of 10 testers were able to give verbal confirmation of the floor and number that the book was held at. The other tester provided confirmation of the floor, but not the classmark. The majority of testers found this information in the availability line in either the brief or full screen. 2 testers found this information in the locations tab. 2 testers needed prompting to give either the floor or the classmark, and the average time taken to find this information was around 19 seconds.
Question 3 - Can you tell us how many copies are out on loan?
Only 3 of the testers managed to complete the task of finding out how many copies were on loan without prompting. 4 testers were unable to complete the task without prompting – many clicked on all tabs except the Locations tab. Some testers commented that they would not look there as it would only tell you where the book was held and not how many we had. Many testers thought that these details would be in the Details or More tabs. There were 3 testers who were unable to complete the task. One of these was due to not recognising the ‘expand’ buttons in the locations tab when there was more than one sublibrary. 3 testers thought that the ‘availability line’ status was clickable and that this was all the information there was available.
Recommendations:
• Re-label the locations tab to something that is more intuitive e.g. one tester suggested ‘availability’ – the ‘locations’ tab has been re-named as ‘Availability’.
• Ensure that if there are multiple sub-libraries that they expand automatically so that all copies are displayed – Ex Libris responded that this is not possible at this time.
• For Ex Libris: possibility of making the availability line status as clickable, taking users into the ‘locations’ tab?
Question 4 - If all copies of this edition were out on loan, can you see if there are any other editions available to borrow and if so how many editions?
9 out of 10 testers completed the task by acknowledging the ‘View x versions’ link and providing confirmation of the number of editions available. Of these, 4 testers had already clicked on the ‘View x versions’ link as part of the previous task and had already noted that it took them to a list of editions of the book. This indicated that we may need to change the labelling of the link to ensure that students would know that it meant ‘editions’ of the book and not ‘copies’. The one tester who did not complete the task was unsure of what an ‘edition’ meant and did not notice the link in either full or brief details screens. During test 7, the ‘View x versions’ link disappeared from the full details screen and so the tester was prompted to return to the brief results. Other testers had seen this link when the system was working correctly.
Recommendations:
• Re-name ‘View x versions’ as ‘View x editions’ – this has been done.
• Ensure that the edition statement appears within the record – we have added this to the ‘Details’ tab.
Question 5 - You are at home and need to find some books and don’t have time to come into the Library. Can you find a list of e-books on the subject of human resource management?
5 of the 10 testers were able to find a list of e-books using the e-book facet on the results screen, 4 testers were able to complete the task with prompting and one tester did not complete the task, but then found the answer whilst completing task 6. Several testers used the ‘Show Online Resources’ link but did not notice that this included journals and e-books. Several testers tried to find an ‘e-books’ option before attempting their search, 3 testers used the advanced search and looked in the format option expecting there to be an ‘e-book’ choice. 3 testers thought that e-books would be in the e-shelf area and were confused by this. 7 of the testers saw the e-book facet – 4 without prompting.
Recommendations:
• Include brief instructions to users on the front page telling them that they can refine their search after retrieving a list of results.
• Re-name ‘e-shelf’ as this caused confusion as testers believed this was where they would find e-resources – this has been re-named as My Favourites.
• Explore further ways of distinguishing between journals and books in the ‘Show Online Resources’ results list
Question 6 - Please try to access one of these items
In this task we wanted to see whether users would be able to see how to open an e-book from the results list, and whether they would be happy with viewing an e-book in the ‘letterbox’ preview screen that is automatically opened when a user clicks on ‘View Online’. There were two potential ways of opening the e-book into a bigger screen – an icon on the right-hand side that would enlarge the e-book into a bigger Primo window (the same view as would happen if a user was in the full-details screen rather than the brief results list view). The second option was a link on the left-hand side that said ‘Open Source in a New Window’ (OSINW) which would open the e-book in a full non-Primo window. Firstly, all of the testers noticed the ‘View Online’ link when either in the brief or full details screens and understood this to mean opening the e-book. Almost all of the testers were unhappy about attempting to view the e-book in the small letterbox view, and many were also reluctant to view the e-book in the bigger Primo window. 5 testers from the brief view clicked on the icon to open the e-book in a bigger screen. Only one tester saw the OSINW link from the brief details screen. Only 4 testers in total saw the OSINW link in either the brief or full details screens. 4 testers attempted to use the internal options within the e-book itself to try and make the e-book bigger – several testers became quite frustrated at not being able to open the e-book in a non-Primo window.
Recommendations:
• Configure Primo to ensure that when clicking on ‘View Online’ the e-book would automatically open in a full-screen, non-Primo window (preferred option) – this is being looked in to by Ex Libris and a fix is due to be released in an upcoming Service Pack.
• If above not possible – swap the links from the OSINW and icon options so that the icon would open a non-Primo window and the OSINW link opened in a Primo window (or preferably both open in a non-Primo window).
• Possibly attempt to re-name OSINW text to something clearer
Tuesday, 21 September 2010
COVPRIMO one minute blog
At Coventry University we have been implementing Primo (which we are branding as 'Locate'), Ex Libris's next generation Library Catalogue.
The main driver behind launching a next generation OPAC was to provide a single user interface for staff and students to use as the starting point for all their Library searches, no matter what type of content they are looking for. This will help provide a more consistent user experience and improve the efficiency of information retrieval. This should help to increase the use of the Library's e-resources. The new system should also help us to provide an attractive interface for our users to access quality information and we believe it will be a viable alternative to systems such as Google, Wikipedia, etc.
The output from our JISC Project will be a report into our implementation of Primo, which should be useful for other institutions considering launching a next generation OPAC and particularly for those deciding to implement Primo themselves. The report will also include an overview of the usability testing we carried out as part of our implementation. This testing was carried out with staff and students to try to identify ways in which we can make our Primo more intuitive and user-friendly. We have then made changes to the system based on these tests. We also gained further student input via a focus group where we discussed what is the key information to include on the Primo home page and how should this be laid out.
The general feedback we received about Primo was overwhelmingly positive, but we did identify issues which needed changes to be made, as they caused problems for several of the users involved in the testing. We have made numerous changes to the system already and we're waiting on Ex Libris to resolve some support issues before making other changes. We are also feeding back the resuts of our tests to Ex Libris and have been told that they will be taking these on board. We hope this means that where we discovered issues that cannot currently be changed Ex Libris will consider adding functionality that we require to future releases of Primo.
Implementation notes
Two IBM rack mounted servers [x3550 M2 8 core processing 16Gb RAM running Red Hat Linux 5.5 64 bit] were purchased and commissioned, one acting as a staging server, the other as the production server. In our case the supplier's software installation was carried out on the production server initially with a copy made to staging fairly late on in the project - normally this procedure is reversed. Project management was handled through the supplier's German office with conference calls established as and when required; these calls tended to increase in length and frequency the closer we moved towards our target switch-to-production (STP) date. On-site staff training was carried out from Ex Libris' UK office and was generally well received by staff.
Once the application had been installed, there were four main thrusts to the implementation work: the design of the end-user interface, taking on board the feedback received as part of the usability study; the testing of data export / import routines from our existing systems (Aleph, MetaLib, SFX and the University's repository product Equella); the establishing of automated data update routines and finally the installation / configuration of third party ancillary software products.
We were aware at the start of the project we had set ourselves an ambitious 'go live' date of early September, hoping to allow subject staff some time to prepare for the forthcoming student induction. In the event a decision has been taken to postpone the launch on two counts. Firstly Shibboleth configuration work is still incomplete and secondly Primo Central functionality is now scheduled to come on stream towards the end of October; Locate will be launched in the new year.
Our JISC project co-ordinator James Fisher leaves Coventry University on Friday 24 September 2010; we wish James well in his new post at the Univeristy of Warwick. Graham Daniels takes over as project contact following James' departure.
The main driver behind launching a next generation OPAC was to provide a single user interface for staff and students to use as the starting point for all their Library searches, no matter what type of content they are looking for. This will help provide a more consistent user experience and improve the efficiency of information retrieval. This should help to increase the use of the Library's e-resources. The new system should also help us to provide an attractive interface for our users to access quality information and we believe it will be a viable alternative to systems such as Google, Wikipedia, etc.
The output from our JISC Project will be a report into our implementation of Primo, which should be useful for other institutions considering launching a next generation OPAC and particularly for those deciding to implement Primo themselves. The report will also include an overview of the usability testing we carried out as part of our implementation. This testing was carried out with staff and students to try to identify ways in which we can make our Primo more intuitive and user-friendly. We have then made changes to the system based on these tests. We also gained further student input via a focus group where we discussed what is the key information to include on the Primo home page and how should this be laid out.
The general feedback we received about Primo was overwhelmingly positive, but we did identify issues which needed changes to be made, as they caused problems for several of the users involved in the testing. We have made numerous changes to the system already and we're waiting on Ex Libris to resolve some support issues before making other changes. We are also feeding back the resuts of our tests to Ex Libris and have been told that they will be taking these on board. We hope this means that where we discovered issues that cannot currently be changed Ex Libris will consider adding functionality that we require to future releases of Primo.
Implementation notes
Two IBM rack mounted servers [x3550 M2 8 core processing 16Gb RAM running Red Hat Linux 5.5 64 bit] were purchased and commissioned, one acting as a staging server, the other as the production server. In our case the supplier's software installation was carried out on the production server initially with a copy made to staging fairly late on in the project - normally this procedure is reversed. Project management was handled through the supplier's German office with conference calls established as and when required; these calls tended to increase in length and frequency the closer we moved towards our target switch-to-production (STP) date. On-site staff training was carried out from Ex Libris' UK office and was generally well received by staff.
Once the application had been installed, there were four main thrusts to the implementation work: the design of the end-user interface, taking on board the feedback received as part of the usability study; the testing of data export / import routines from our existing systems (Aleph, MetaLib, SFX and the University's repository product Equella); the establishing of automated data update routines and finally the installation / configuration of third party ancillary software products.
We were aware at the start of the project we had set ourselves an ambitious 'go live' date of early September, hoping to allow subject staff some time to prepare for the forthcoming student induction. In the event a decision has been taken to postpone the launch on two counts. Firstly Shibboleth configuration work is still incomplete and secondly Primo Central functionality is now scheduled to come on stream towards the end of October; Locate will be launched in the new year.
Our JISC project co-ordinator James Fisher leaves Coventry University on Friday 24 September 2010; we wish James well in his new post at the Univeristy of Warwick. Graham Daniels takes over as project contact following James' departure.
Monday, 6 September 2010
Focus Group - Primo Home Page
In order to get student feedback about what information should be on our Primo home page and the layout of the home page we arranged a focus group with ten students at the end of August. To get our participants we first emailed all of our volunteers who had given us their email addresses back in June. We had a much lower response rate than when we were arranging the usability tests. So to get more participants we went round the Library asking students if they would be available to come along to our group. We soon reached our target of ten students, and were very pleased that they all turned up at the appointed time.
The session was extremely productive with all the students contributing and lots of ideas being discussed. To start the session the three members of Library staff introduced themselves and then the ten students did the same and told the group what they were studying. To get everyone warmed up we asked them to tell us what they do first of all when they get an assignment and have to find relevant information. This produced a lively discussion and despite a number of students using Wikipedia and Google it was reassuring to find that everyone made use of the Library’s subscribed resources too and most seemed to use MetaLib to get to the databases. There was a quick debate around the pros and cons of Wikipedia. Most students agreed that they shouldn’t reference it and many had been given strict instructions about this by their lecturers. However, a number of students found it a useful starting point and used it to find search terms about their topic to enter into the Library databases. This was an interesting aside and seemed to get everyone involved from the very start of the session.
We then gave a quick overview of what to expect from the rest of the session. We also showed the group what the Coventry Primo currently looks like and did a quick demo of a search. We had some immediate positive feedback at this point about the look of the Primo interface. It was described by one user as “very easy on the eye.” Then we told them about the usability studies that had already taken place, and highlighted one of the changes that we had made because of these studies.
We then asked the participants to suggest as many ideas as they could think of that they would like to see on the home page. The list of suggestions that the group came up with and agreed would be useful on the home page was as follows:
· Announcements (e.g. work going on in the Library at the moment)
· Room booking / availability
· Search hints, tips / Search examples
· Introduction to Primo – Tutorial
· Contacts, Profiles, Who’s Who, Who’s responsible for certain subject areas
· Advertising for the University as a whole – Student Centre, Social Events, etc.
· Opening times
· Advanced Search
· Contact IT Services for help with your password, or with signing in
· Feedback section for Primo or for anything to do with the Library
· Ask a Librarian
· Latest books added to Library stock in different subject areas
· Passwords help, change password information
· Image of the Library (maybe as a watermark with links on top)
· A video with search hints
· Autocomplete functionality when typing searches
· Link to the Centre for Academic Writing & referencing information
· Library FAQs
· CU Portal link
· CUOnline / Moodle link
We then wrote each of the suggestions on post-its and worked on a flip-chart with a hand drawn version of the Primo home page. This had five areas on it. These were the header area which includes the search box, and the footer area. These two areas will remain fixed as users carry out searches. The rest of the home page is divided into three parts: a top left and top right section and a bottom section which goes across the whole page. The group suggested whereabouts on the home page was the best place to put each of their suggestions. There was quite a bit of debate around where was the best place to put each suggestion and some ideas were even rejected and removed from the list. In the end a consensus was reached with similar items being grouped together.
It was suggested that Announcements could appear in a ‘ticker’ format scrolling across the screen. This was something to look at in the future. There was a big debate around where on the page they should appear. It was agreed this should be limited to just Library announcements, and nothing else, and maybe just the top few announcements and headlines only to keep it brief. Eventually it was agreed that links to announcements could go on the top right of the page, along with new books added to stock and room bookings information.
Search hints should be at the top on the left under the search box. Also a short introduction to Primo should be here and possibly also a link to a video introduction, which is something else to look at in the future. An Ask a Librarian link should be in this area too. There was agreement to add a link to the text at the end of the search hints.
The Ask a Librarian link should also be in the header, so it is always available. The header and footer stay on the screen when a search is carried out, so would always be available, whereas the rest of the home page would be replaced by search results, etc. There was also consensus that a link to Library Passwords information should be in this section.
Contact information for Subject Librarians could go in the footer as this is a common place to find contacts. Links to IT Services & general ITS Passwords information could also go in this section, along with Library FAQs.
There was discussion over where to put the link to the student portal (CU Portal). Suggestions were to put this in the footer, but also to put a link to this at the top of the screen. Eventually we agreed to make the Coventry University logo at the top of the screen a link to CU Portal as this is the most likely place you would want to link to from Primo and also put another link in the footer.
It was suggested that the opening hours could go in the bottom section of the home page along with links to the Centre for Academic Writing and referencing.
The group thought it would be good if an image of the Library could be included on the page, possibly as a watermark. However, it was agreed that this should only be added if it didn’t distract users from the text on the home page.
We then looked at three other Primo Version 3 home pages, that are already live, and the group gave us input into what they liked and didn’t like about these sites.
After looking at the other sites it was agreed that the amount of text on the Coventry Primo home page should be kept to a minimum. One comment was “The simpler the better.” Ideally, the group wanted headings in bold that you could click on to make the information beneath those headers display. This would mean that initially there would just be headings on the page, which would prevent it from being too cluttered. However, it needs to be clear that this is what will happen if you click on the heading. The information also needs to be spread out enough to fill the screen.
The group also commented that the out-of-the-box image of books on a shelf, that displays on several Primo sites really drew the eye. However, this was probably the least important thing on the page. They still felt it could be good to include a watermarked image of our Library. However, we would need to be careful this didn’t detract from the information on the page if we included this.
Revisiting the mock-up of the home page it was again agreed to keep similar items grouped together but where possible to just display a general heading which you can click on in order to display the links beneath that heading. A picture of the final mock-up can be seen below:
One other point of interest that was raised several times during the session was the expectation of personalised information for individual users. Participants wanted information about previous searches, books that they have looked for or used before and new books in the Library relevant to their subject. We explained that much of this information is available in the area that we have named ‘My Favourites’. We didn’t go into too much more detail about this as we were focussing on the home page for all users, initially accessing it as a guest. However, it was clear that personalisation was something that should be looked at further in the future.
Overall it was an incredibly useful session. We will now sit down with the Library Systems Manager who is working on these kinds of developments in Primo and try to turn the comments into reality.
The session was extremely productive with all the students contributing and lots of ideas being discussed. To start the session the three members of Library staff introduced themselves and then the ten students did the same and told the group what they were studying. To get everyone warmed up we asked them to tell us what they do first of all when they get an assignment and have to find relevant information. This produced a lively discussion and despite a number of students using Wikipedia and Google it was reassuring to find that everyone made use of the Library’s subscribed resources too and most seemed to use MetaLib to get to the databases. There was a quick debate around the pros and cons of Wikipedia. Most students agreed that they shouldn’t reference it and many had been given strict instructions about this by their lecturers. However, a number of students found it a useful starting point and used it to find search terms about their topic to enter into the Library databases. This was an interesting aside and seemed to get everyone involved from the very start of the session.
We then gave a quick overview of what to expect from the rest of the session. We also showed the group what the Coventry Primo currently looks like and did a quick demo of a search. We had some immediate positive feedback at this point about the look of the Primo interface. It was described by one user as “very easy on the eye.” Then we told them about the usability studies that had already taken place, and highlighted one of the changes that we had made because of these studies.
We then asked the participants to suggest as many ideas as they could think of that they would like to see on the home page. The list of suggestions that the group came up with and agreed would be useful on the home page was as follows:
· Announcements (e.g. work going on in the Library at the moment)
· Room booking / availability
· Search hints, tips / Search examples
· Introduction to Primo – Tutorial
· Contacts, Profiles, Who’s Who, Who’s responsible for certain subject areas
· Advertising for the University as a whole – Student Centre, Social Events, etc.
· Opening times
· Advanced Search
· Contact IT Services for help with your password, or with signing in
· Feedback section for Primo or for anything to do with the Library
· Ask a Librarian
· Latest books added to Library stock in different subject areas
· Passwords help, change password information
· Image of the Library (maybe as a watermark with links on top)
· A video with search hints
· Autocomplete functionality when typing searches
· Link to the Centre for Academic Writing & referencing information
· Library FAQs
· CU Portal link
· CUOnline / Moodle link
We then wrote each of the suggestions on post-its and worked on a flip-chart with a hand drawn version of the Primo home page. This had five areas on it. These were the header area which includes the search box, and the footer area. These two areas will remain fixed as users carry out searches. The rest of the home page is divided into three parts: a top left and top right section and a bottom section which goes across the whole page. The group suggested whereabouts on the home page was the best place to put each of their suggestions. There was quite a bit of debate around where was the best place to put each suggestion and some ideas were even rejected and removed from the list. In the end a consensus was reached with similar items being grouped together.
It was suggested that Announcements could appear in a ‘ticker’ format scrolling across the screen. This was something to look at in the future. There was a big debate around where on the page they should appear. It was agreed this should be limited to just Library announcements, and nothing else, and maybe just the top few announcements and headlines only to keep it brief. Eventually it was agreed that links to announcements could go on the top right of the page, along with new books added to stock and room bookings information.
Search hints should be at the top on the left under the search box. Also a short introduction to Primo should be here and possibly also a link to a video introduction, which is something else to look at in the future. An Ask a Librarian link should be in this area too. There was agreement to add a link to the text at the end of the search hints.
The Ask a Librarian link should also be in the header, so it is always available. The header and footer stay on the screen when a search is carried out, so would always be available, whereas the rest of the home page would be replaced by search results, etc. There was also consensus that a link to Library Passwords information should be in this section.
Contact information for Subject Librarians could go in the footer as this is a common place to find contacts. Links to IT Services & general ITS Passwords information could also go in this section, along with Library FAQs.
There was discussion over where to put the link to the student portal (CU Portal). Suggestions were to put this in the footer, but also to put a link to this at the top of the screen. Eventually we agreed to make the Coventry University logo at the top of the screen a link to CU Portal as this is the most likely place you would want to link to from Primo and also put another link in the footer.
It was suggested that the opening hours could go in the bottom section of the home page along with links to the Centre for Academic Writing and referencing.
The group thought it would be good if an image of the Library could be included on the page, possibly as a watermark. However, it was agreed that this should only be added if it didn’t distract users from the text on the home page.
We then looked at three other Primo Version 3 home pages, that are already live, and the group gave us input into what they liked and didn’t like about these sites.
After looking at the other sites it was agreed that the amount of text on the Coventry Primo home page should be kept to a minimum. One comment was “The simpler the better.” Ideally, the group wanted headings in bold that you could click on to make the information beneath those headers display. This would mean that initially there would just be headings on the page, which would prevent it from being too cluttered. However, it needs to be clear that this is what will happen if you click on the heading. The information also needs to be spread out enough to fill the screen.
The group also commented that the out-of-the-box image of books on a shelf, that displays on several Primo sites really drew the eye. However, this was probably the least important thing on the page. They still felt it could be good to include a watermarked image of our Library. However, we would need to be careful this didn’t detract from the information on the page if we included this.
Revisiting the mock-up of the home page it was again agreed to keep similar items grouped together but where possible to just display a general heading which you can click on in order to display the links beneath that heading. A picture of the final mock-up can be seen below:
One other point of interest that was raised several times during the session was the expectation of personalised information for individual users. Participants wanted information about previous searches, books that they have looked for or used before and new books in the Library relevant to their subject. We explained that much of this information is available in the area that we have named ‘My Favourites’. We didn’t go into too much more detail about this as we were focussing on the home page for all users, initially accessing it as a guest. However, it was clear that personalisation was something that should be looked at further in the future.
Overall it was an incredibly useful session. We will now sit down with the Library Systems Manager who is working on these kinds of developments in Primo and try to turn the comments into reality.
Friday, 3 September 2010
Usability Testing Primo
About a month ago we carried out our usability tests on the out-of-the-box version of Primo. This had been set up to use data from our Aleph Library Management System test server. We also had data from MetaLib, our Federated Search Tool and SFX our Open URL Resolver harvested into Primo. We wanted to get input from staff and students on things that they found unintuitive, so that we could try to make improvements to the system before launching it.
Before starting testing we visited the University’s usability lab and talked things through with the technician. We also enlisted the help of an experienced usability tester, a PhD student from the University who had spent a lot of time running tests as part of her research. She gave us plenty of helpful advice and even attended our pilot study to give us feedback on ways to improve our test before running it with students and academic staff.
We ran the pilot study using a member of Library staff who had not seen our Primo installation at all, so had no preconceptions. Carrying out this pilot enabled us to familiarise ourselves with the environment in the usability lab and also to check that our questions all seemed to work and make sense to the tester. We did identify some changes to make to the questions before carrying out the testing for real.
We recruited 8 students and 2 members of academic staff to carry out the usability tests. We had obtained about 40 students’ contact details earlier in the project when carrying out a survey on what to call our Primo. This meant we were able to send emails to the volunteers with the incentive of paying for their time if they came in and worked with us. Although it was an unfortunate time to carry out the tests right in the middle of the summer vacation we still managed to recruit our 8 students without too many problems. Unsurprisingly we didn’t receive responses from a number of emails, but we were able to continue sending out requests until we had enough volunteers. Pleasingly, although we had to rearrange a couple of the tests at the request of the students everyone who volunteered did attend and arrived on time. The members of staff that we recruited were regular Library users who were suggested by the Subject Librarians who knew them.
We allowed an hour to carry out each test and to have a discussion afterwards and found that a number of different themes emerged, which helped us identify changes we wanted to make in order to make the system more user-friendly. Firstly there were things to do with the appearance of the interface, which we felt could be changed to make it more intuitive. For example, after carrying out a search the facets available to refine that search are displayed in a column on the left-hand-side of the screen. However, there is not much differentiation between these facets and the search results that display in the centre of the screen. Everything is shown on the same white background and there is only a faint line between the facets and the results. This meant that a number of our testers didn’t spot the facets for quite some time. Once they did spot them they did understand what they were for and used them intuitively to narrow down their search. We had a number of pleasing comments about how obvious it was what the facets did and also comparisons with Tesco Online and Amazon in the way that they worked. This showed how important it is to make the facets prominent to the users as they did find them useful once they noticed them. Therefore, in order to try to make them stand out more we added shading to the facets to differentiate them from the rest of the page and made the headings bolder. We also made the line between the facets and the results more defined. We believe this will make the facets easier to spot.
The second theme that emerged was to do with terminology that was used. An example of this is that the personalised area of Primo where users can save search results or previous searches, etc. was originally called the e-Shelf. We had a number of testers who thought that this would be where they would find e-books, which does have a certain logic to it: An e-book would be on an e-Shelf. Several other testers were also confused by the terminology and kept returning to this area, even though it wasn’t relevant to any of the tasks they have been set. Therefore, in order to try to avoid this confusion we have renamed the e-Shelf as ‘My Favourites’ as we believe this is clearer.
Another theme that emerged was to do with the functionality of Primo. For example when Primo finds multiple editions of an item it groups them all together under the Functional Requirements for Bibliographic Records (FRBR) rules. However, when searching by relevance it always prioritises the online version, even if this isn’t the most recent. In our experience with e-books, publishers often only make an electronic version of a title available some time after they have published the print version. This delay can be 6 months or it could be a year or more. This means that we often have new print editions of essential books available in the Library well before we are able to obtain an electronic version. We will often have previous print and electronic editions. This means that when Primo groups these records into a FRBR group the edition it chooses to display in the list of search results is the old electronic edition. The user won’t find out that we hold the most recent edition until they click on the FRBR group and examine the list of other available editions. When we asked Ex Libris if we could continue to search by relevance but make sure that the most recent edition of a book displays in the results list we were told this wasn’t standard functionality. Therefore, we had to raise a call to get our request investigated further and it’s possible that we won’t be able to make this change until a future version of Primo is released. We are waiting in hope for Ex Libris to investigate our request as we would like to make this change before we go live with Primo.
Most of the changes that we identified through the usability tests could be put into one or more of these categories. Since the tests both Paul here in the Library and Lee from Ex Libris have been working very hard to configure our Primo based on the results. It has already changed a great deal, but there is still more work to do. We have a consultancy day with Ex Libris here at Coventry next week, so we are hoping that we will be able to clear up many of the outstanding issues then.
Before starting testing we visited the University’s usability lab and talked things through with the technician. We also enlisted the help of an experienced usability tester, a PhD student from the University who had spent a lot of time running tests as part of her research. She gave us plenty of helpful advice and even attended our pilot study to give us feedback on ways to improve our test before running it with students and academic staff.
We ran the pilot study using a member of Library staff who had not seen our Primo installation at all, so had no preconceptions. Carrying out this pilot enabled us to familiarise ourselves with the environment in the usability lab and also to check that our questions all seemed to work and make sense to the tester. We did identify some changes to make to the questions before carrying out the testing for real.
We recruited 8 students and 2 members of academic staff to carry out the usability tests. We had obtained about 40 students’ contact details earlier in the project when carrying out a survey on what to call our Primo. This meant we were able to send emails to the volunteers with the incentive of paying for their time if they came in and worked with us. Although it was an unfortunate time to carry out the tests right in the middle of the summer vacation we still managed to recruit our 8 students without too many problems. Unsurprisingly we didn’t receive responses from a number of emails, but we were able to continue sending out requests until we had enough volunteers. Pleasingly, although we had to rearrange a couple of the tests at the request of the students everyone who volunteered did attend and arrived on time. The members of staff that we recruited were regular Library users who were suggested by the Subject Librarians who knew them.
We allowed an hour to carry out each test and to have a discussion afterwards and found that a number of different themes emerged, which helped us identify changes we wanted to make in order to make the system more user-friendly. Firstly there were things to do with the appearance of the interface, which we felt could be changed to make it more intuitive. For example, after carrying out a search the facets available to refine that search are displayed in a column on the left-hand-side of the screen. However, there is not much differentiation between these facets and the search results that display in the centre of the screen. Everything is shown on the same white background and there is only a faint line between the facets and the results. This meant that a number of our testers didn’t spot the facets for quite some time. Once they did spot them they did understand what they were for and used them intuitively to narrow down their search. We had a number of pleasing comments about how obvious it was what the facets did and also comparisons with Tesco Online and Amazon in the way that they worked. This showed how important it is to make the facets prominent to the users as they did find them useful once they noticed them. Therefore, in order to try to make them stand out more we added shading to the facets to differentiate them from the rest of the page and made the headings bolder. We also made the line between the facets and the results more defined. We believe this will make the facets easier to spot.
The second theme that emerged was to do with terminology that was used. An example of this is that the personalised area of Primo where users can save search results or previous searches, etc. was originally called the e-Shelf. We had a number of testers who thought that this would be where they would find e-books, which does have a certain logic to it: An e-book would be on an e-Shelf. Several other testers were also confused by the terminology and kept returning to this area, even though it wasn’t relevant to any of the tasks they have been set. Therefore, in order to try to avoid this confusion we have renamed the e-Shelf as ‘My Favourites’ as we believe this is clearer.
Another theme that emerged was to do with the functionality of Primo. For example when Primo finds multiple editions of an item it groups them all together under the Functional Requirements for Bibliographic Records (FRBR) rules. However, when searching by relevance it always prioritises the online version, even if this isn’t the most recent. In our experience with e-books, publishers often only make an electronic version of a title available some time after they have published the print version. This delay can be 6 months or it could be a year or more. This means that we often have new print editions of essential books available in the Library well before we are able to obtain an electronic version. We will often have previous print and electronic editions. This means that when Primo groups these records into a FRBR group the edition it chooses to display in the list of search results is the old electronic edition. The user won’t find out that we hold the most recent edition until they click on the FRBR group and examine the list of other available editions. When we asked Ex Libris if we could continue to search by relevance but make sure that the most recent edition of a book displays in the results list we were told this wasn’t standard functionality. Therefore, we had to raise a call to get our request investigated further and it’s possible that we won’t be able to make this change until a future version of Primo is released. We are waiting in hope for Ex Libris to investigate our request as we would like to make this change before we go live with Primo.
Most of the changes that we identified through the usability tests could be put into one or more of these categories. Since the tests both Paul here in the Library and Lee from Ex Libris have been working very hard to configure our Primo based on the results. It has already changed a great deal, but there is still more work to do. We have a consultancy day with Ex Libris here at Coventry next week, so we are hoping that we will be able to clear up many of the outstanding issues then.
Friday, 16 July 2010
Naming our Primo installation
One of the preliminary discussions that we had early on in our project was what to call our Primo installation. It soon became clear that there were two distinct options. We could either try to retain some continuity or we could choose a completely new name to indicate that this is a brand new system.
As we are planning to use Primo to completely replace our Library Catalogue and eLibrary (MetaLib) interfaces we could try to retain some continuity by using a slightly amended form of these names. To this end Library Catalogue Plus and eLibrary Plus were suggested. Certainly we have seen examples of other Primo sites where the word Plus has been added to the current Library Catalogue name, so this felt like a valid possibility.
The alternative option was to think of a brand new name and after much discussion we decided on two possibilities. These were SearchLight and LOCATe. We thought SearchLight could indicate that you were shining a light onto your search in order to find information. The name LOCATe could stand for Library Online Catalogue, and of course the word locate also means find. It was agreed that if either of these names were selected we would also need a strap line which would summarise what the system does.
As we were having these discussions just before the end of term we decided to try to get some student input into which name our users preferred. In order to try to reach as many people as possible we used two methods to get feedback. Firstly, we went around the Library and asked users if they could spare a few minutes to give us their views. To those that were happy to do this we explained briefly about the plan to introduce Primo and asked them which name they liked best. Secondly, we created a SurveyMonkey poll where we again explained about Primo and asked which of the four names respondents preferred.
We received just under 150 responses, split very evenly between online and face-to-face. Unfortunately, the results weren’t completely conclusive, although there were two names that came out ahead. These were eLibrary Plus and LOCATe and both had a similar number of votes. As term was ending and there wasn’t time to do another student poll it was decided to survey the Library staff and ask them to choose between these two options. This time there was a more conclusive result and the name LOCATe was chosen.
A very useful side benefit of carrying out the user survey about the name was that we also asked for volunteers to take part in our usability testing over the summer. We knew this was likely to take place in July or August, so we asked everyone who gave us input about the name if they lived close enough to the University to come in over summer and if so whether they were interested in taking part in some testing if there was a financial incentive. As a result of this we got 40 volunteers, all of whom gave us their email addresses and said we could contact them when we were arranging the testing. This was a big relief as one of our concerns had been how we would contact users about the tests during the quiet summer period.
As we are planning to use Primo to completely replace our Library Catalogue and eLibrary (MetaLib) interfaces we could try to retain some continuity by using a slightly amended form of these names. To this end Library Catalogue Plus and eLibrary Plus were suggested. Certainly we have seen examples of other Primo sites where the word Plus has been added to the current Library Catalogue name, so this felt like a valid possibility.
The alternative option was to think of a brand new name and after much discussion we decided on two possibilities. These were SearchLight and LOCATe. We thought SearchLight could indicate that you were shining a light onto your search in order to find information. The name LOCATe could stand for Library Online Catalogue, and of course the word locate also means find. It was agreed that if either of these names were selected we would also need a strap line which would summarise what the system does.
As we were having these discussions just before the end of term we decided to try to get some student input into which name our users preferred. In order to try to reach as many people as possible we used two methods to get feedback. Firstly, we went around the Library and asked users if they could spare a few minutes to give us their views. To those that were happy to do this we explained briefly about the plan to introduce Primo and asked them which name they liked best. Secondly, we created a SurveyMonkey poll where we again explained about Primo and asked which of the four names respondents preferred.
We received just under 150 responses, split very evenly between online and face-to-face. Unfortunately, the results weren’t completely conclusive, although there were two names that came out ahead. These were eLibrary Plus and LOCATe and both had a similar number of votes. As term was ending and there wasn’t time to do another student poll it was decided to survey the Library staff and ask them to choose between these two options. This time there was a more conclusive result and the name LOCATe was chosen.
A very useful side benefit of carrying out the user survey about the name was that we also asked for volunteers to take part in our usability testing over the summer. We knew this was likely to take place in July or August, so we asked everyone who gave us input about the name if they lived close enough to the University to come in over summer and if so whether they were interested in taking part in some testing if there was a financial incentive. As a result of this we got 40 volunteers, all of whom gave us their email addresses and said we could contact them when we were arranging the testing. This was a big relief as one of our concerns had been how we would contact users about the tests during the quiet summer period.
Tuesday, 15 June 2010
Budget
JISC require that an outline budget is posted, so here is the budget for the covprimo project:
Projected Timeline, Workplan & Methodology
The following chart details the main work packages as identified by Ex Libris in their Primo Implementation project plan and the timescales they are working to. Local tasks such as the usability testing and the launch and publicity have then been planned to take place at appropriate times.
Tuesday, 25 May 2010
Project Team Relationships and End User Engagement
The project team at Coventry consists of the following members:
Caroline Rock – University Librarian (JISC Project Champion)
James Fisher – E-Resources Co-ordinator (JISC Project Manager)
Graham Daniels – Head of Electronic and Network Services (Primo Implementation Project Manager)
Sandy Forster – Head of Bibliographic Support (providing input on the management of bibliographic records from the Aleph Library Management System)
Paul Smith – Library Systems Manager (carrying out most of the technical developments on the Library side)
Theresa Morley – Assistant Subject Librarian (giving a user-focussed view to the project) Carolyn Haresign – Head of Academic Services (also giving a user-focussed view and providing experience of running focus groups)
Joanne Marsh - Repository Development Officer (providing input on integrating the University’s repository (Equella) with Primo)
The majority of the Coventry project team can be seen in these photos.
Caroline Rock – University Librarian (JISC Project Champion)
James Fisher – E-Resources Co-ordinator (JISC Project Manager)
Graham Daniels – Head of Electronic and Network Services (Primo Implementation Project Manager)
Sandy Forster – Head of Bibliographic Support (providing input on the management of bibliographic records from the Aleph Library Management System)
Paul Smith – Library Systems Manager (carrying out most of the technical developments on the Library side)
Theresa Morley – Assistant Subject Librarian (giving a user-focussed view to the project) Carolyn Haresign – Head of Academic Services (also giving a user-focussed view and providing experience of running focus groups)
Joanne Marsh - Repository Development Officer (providing input on integrating the University’s repository (Equella) with Primo)
The majority of the Coventry project team can be seen in these photos.
The project team from Ex Libris consists of the following members:
Georg Heiming, Ex Libris, Germany (Project Manager)
Lee Smith, Ex Libris, UK (working on Coventry Primo implementation for Ex Libris)
Leonard Wright, Ex Libris, UK (working with Lee on Coventry Primo implementation for Ex Libris)
Engagement with end users has already started with a survey to consider potential names for the CU Primo product when it is launched. Once the initial version of Primo has been made available usability testing will be carried out and the results of this testing will be used to inform changes that will be made to make the interface more intuitive. During the survey about the name for Primo, respondents were asked if they would like to volunteer to take part in usability testing over the summer and 40 people volunteered, leaving their details so they can be contacted about taking part in the testing.
Georg Heiming, Ex Libris, Germany (Project Manager)
Lee Smith, Ex Libris, UK (working on Coventry Primo implementation for Ex Libris)
Leonard Wright, Ex Libris, UK (working with Lee on Coventry Primo implementation for Ex Libris)
Engagement with end users has already started with a survey to consider potential names for the CU Primo product when it is launched. Once the initial version of Primo has been made available usability testing will be carried out and the results of this testing will be used to inform changes that will be made to make the interface more intuitive. During the survey about the name for Primo, respondents were asked if they would like to volunteer to take part in usability testing over the summer and 40 people volunteered, leaving their details so they can be contacted about taking part in the testing.
Tuesday, 18 May 2010
Intellectual Property Rights
As required by JISC the case study report and all other outputs will be published using the Creative Commons Attribution-Share Alike UK 2.0: England and Wales license.
Managing risks and success
The intention is to launch Primo in time for the start of the new academic year in September 2010, although it will continue to be developed and tested after the launch. If for some reason this is not achieved the contingency plan is to carry out a soft launch at a slightly later date, making Primo available alongside the current Library Catalogue and MetaLib systems. These other interfaces will then be removed later on, leaving Primo as the single user interface, for searching for Library material.
The risks that have been identified relating to the project are listed below, along with the actions required to mitigate against those risks.
1. The risk of losing key personnel can be reduced by building expertise across the team.
2. The risk of delays due to inter-departmental dependencies can be reduced by ensuring effective internal communication and the inclusion of relevant staff in the team.
3. The risk of delays due to supplier dependencies can be reduced by agreeing the implementation timescale in the contract.
4. The risk of the unavailability of stakeholders for consultation and feedback can be reduced by effective project planning and the offering of incentives and by engaging with stakeholders early in the project.
Since starting the project the team has realised that there is a risk to authenticating via Shibboleth with the Aleph Library Management System, which the technical members of the team are working to resolve.
It is hoped that Primo will be very popular with the users at Coventry. There are no major problems anticipated as a result of this. However, we do need to be aware of the resource implications of continuing to develop the product and ensure that there is staff time set aside to do this.
Similarly, if the case study report is well received this shouldn’t cause any problems.
The risks that have been identified relating to the project are listed below, along with the actions required to mitigate against those risks.
1. The risk of losing key personnel can be reduced by building expertise across the team.
2. The risk of delays due to inter-departmental dependencies can be reduced by ensuring effective internal communication and the inclusion of relevant staff in the team.
3. The risk of delays due to supplier dependencies can be reduced by agreeing the implementation timescale in the contract.
4. The risk of the unavailability of stakeholders for consultation and feedback can be reduced by effective project planning and the offering of incentives and by engaging with stakeholders early in the project.
Since starting the project the team has realised that there is a risk to authenticating via Shibboleth with the Aleph Library Management System, which the technical members of the team are working to resolve.
It is hoped that Primo will be very popular with the users at Coventry. There are no major problems anticipated as a result of this. However, we do need to be aware of the resource implications of continuing to develop the product and ensure that there is staff time set aside to do this.
Similarly, if the case study report is well received this shouldn’t cause any problems.
Tuesday, 11 May 2010
Benefits from the project to Coventry University and the wider community
The introduction of Primo should benefit the users at Coventry by providing a simpler search experience and making information retrieval easier as previously outlined. It will also simplify searching by providing a single point of access when linking from other student systems, such as the VLE and student portal. This should help to improve student satisfaction. The new interface should also increase confidence that the best use is being made of the Library’s resources.
The case study and usability testing should benefit the wider community by deepening the understanding of the information seeking behaviour of the ‘google generation’, piloting Web 2.0 technologies, testing the adequacy of harvestable metadata and testing the interoperability of related information systems. It should also be of benefit to the Primo community who may want to use the report to inform changes to their own Primo interfaces.
The case study and usability testing should benefit the wider community by deepening the understanding of the information seeking behaviour of the ‘google generation’, piloting Web 2.0 technologies, testing the adequacy of harvestable metadata and testing the interoperability of related information systems. It should also be of benefit to the Primo community who may want to use the report to inform changes to their own Primo interfaces.
Thursday, 6 May 2010
Project aims, objectives and outputs
This is a blog about the Coventry University Primo Implementation Project. This is a project to implement Ex Libris’s product Primo as the single user interface for searching for all Library material (books, journals, articles, DVDs, streamed video, etc.) at Coventry University. The new interface will search the Library Catalogue, the Library’s federated search tool (MetaLib), pre-harvested metadata from certain Library databases and the University’s Institutional Repository (Equella). This should help to deliver the objectives of the project which are:
1. To provide a consistent user experience when retrieving Library information from a range of databases and other data sources (e.g. the Library Catalogue, MetaLib and Equella).
2. To improve the efficiency of information retrieval, using web 2.0 functionality
3. To increase the use of the Library’s e-resources in order to improve return on investment. The enhanced and more intuitive interface should lead to an increase in the identification and use of the e-resources.
4. To contribute to the development of a ‘community of learners’ by offering social networking tools appropriate to an HE information environment
The experience of implementing and evaluating (with usability testing) Primo at Coventry will be written up as a case study to be submitted to JISC and shared with the Library community.
1. To provide a consistent user experience when retrieving Library information from a range of databases and other data sources (e.g. the Library Catalogue, MetaLib and Equella).
2. To improve the efficiency of information retrieval, using web 2.0 functionality
3. To increase the use of the Library’s e-resources in order to improve return on investment. The enhanced and more intuitive interface should lead to an increase in the identification and use of the e-resources.
4. To contribute to the development of a ‘community of learners’ by offering social networking tools appropriate to an HE information environment
The experience of implementing and evaluating (with usability testing) Primo at Coventry will be written up as a case study to be submitted to JISC and shared with the Library community.
Subscribe to:
Posts (Atom)
