Primo Usability Testing - Details Post #2

This is the second of our 3 posts giving a detailed write-up of our usability testing.

Section 2 - Searching for known journals and journal articles

Question 7 - Your lecturer has told you to read an article in the most recent issue of the journal, the Harvard Business Review. Check to see if this is available in print, electronically or in both formats.

In this task we wanted to see how the testers would search for an individual journal. There are clear options for this in the library catalogue and eLibrary; however Primo does not explicitly offer an option for this at first glance. Only 2 of the 10 testers were able to correctly identify that we had both print and electronic access to the journal. The majority of testers saw ‘Online access’ and ‘Media format: E-Journal’ and suggested that we only had electronic access to the journal. Only two testers clicked on the Locations tab which is where they would find details of the print access. One tester found the print access by clicking on the ‘More’ tab and noticing the ‘Check print availability’ option in the SFX menu – however they did not verbally confirm if there was electronic access. 4 of the testers tried to search for the journal in the ‘Articles etc’ tab as they assumed that this was where they would find journals. Some of these were prompted to look at the ‘etc’ part of the tab name – one tester stated that they thought this was very unprofessional terminology to use in a higher education environment. Two of the testers used the Advanced Search option in the Books etc tab and chose ‘Journal’ from the format. Other testers commented that they had previously used the Journal Search or Find e-journal options with the Library Catalogue and eLibrary. This indicates that the testers were looking for ‘somewhere’ to search for journals and wouldn’t necessarily use the Books etc tab for this purpose.
Recommendations:
• Re-name the ‘Books etc’ tab to include the word ‘journal’ and create a more professional-sounding name – this would indicate to users more clearly where they should search for different material types – this has been re-named ‘Books, Journals, Media & More’.
• Find a way of making print journal access more visible to the user – possibly re-naming the Locations tab could help with this.
• Check with Ex Libris to see if different text can be displayed in the ‘Availability Status’ line (would also help with e-books – where print availability is also hidden) – Ex Libris responded that a different text can be displayed – we are still looking in alternative wording.
• Change the information in the Media Format section of the ‘Details’ tab to include both print and electronic access – if this is not possible would removing it altogether be an option? – we have removed the Media format option from this tab.

Question 7a - What years are available and from where?

Out of 9 testers who attempted the question, only 2 were able to correctly identify the years of access as from 1952 (print) and from 1922 (electronically). The majority of testers thought that we only had access to one year of the journal – 1922. This date was mostly identified in the results list, the details tab or in the sfx menu. 1 tester did not verbally conform any dates at all, despite scrolling through a list of journal issues in EBSCO. Unfortunately, due to the lack of recognition of print availability (as shown in question 7) it was not possible to effectively evaluate the information shown in the journals locations tab, although the testers who found it were confused by the amount of information on the screen and took some time to notice the Summary Holdings statement showing the years available.
Recommendations:
• If possible include the journal ‘from and to dates’ in the brief display and full details tabs (not possible due to cataloguing implications)
• If the above is not possible, then remove the date entirely from these sections as it could be deemed to be very misleading – we have removed the date for journals.
• Try to remove all of the information in the journals locations tab to make the Summary Holdings statement stand out more prominently (unfortunately this is unable to be done when using Opac via Primo (OvP) as this also removes the holdings information for books. Turning the information off, also removes the Summary Holdings statement).

Question 8 - You need to find general journal articles on the subject of customer service training – how would you carry out this search? (You will need to sign in with your Coventry University username and password before starting the search)

This question was included to see how easy students would find the Articles tab to locate and use. At the time of testing the Articles tab had not been set up quite in the way that was needed and so inevitably there was quite a lot of prompting to ensure testers came up with a list of results that would be useful for the follow-up question (#9). Only half of the testers found the Articles tab without prompting. Of the 5 remaining testers, 2 really struggled to find somewhere to search for articles, and 1 tester attempted to use the Journal and Article options within the Books tab advanced search. All testers found the sign-in link straight away which was very encouraging. All testers needed prompting to use the General Quickset, however some did look in this drop-down menu before prompting which indicates that if there had been subject-based quicksets there, it may have been more intuitive. 2 testers encountered a blank screen after having signed in after trying to conduct their search - behind this Primo was actually conducting the search, however there was no indication of this and caused some confusion.
Recommendations:
• Provide instructions on the front page of Primo to indicate what students would be able to find in the Articles tab.
• For Ex Libris: ensure that a blank screen is not displayed when a search is conducted when sign in happens after search terms are entered – Ex Libris have been looking in to this, but it has yet to be fixed.
• Insert text into Quickset box asking students to select a subject area – this has been done.
• More usability testing once the Articles tab is set up with our quicksets (dependent on Primo Central installation).

Question 9 - Please try to view one of the articles you have found.

This question was to see how testers would attempt to get to the full text of the article. Ex Libris had named the tab which linked to our SFX installation as ‘Services’ and we wanted to see if this was an intuitive place for the testers to click. The ‘Services’ tab was located in the same place that the ‘View online’ tab was from the Books etc search tab. Each article on the results list came up as ‘No Full Text’ even if we had access – this is related to the information coming from SFX and was incorrect due to not having implemented the most recent version of SFX. The ‘No Full Text’ information in the availability line misled many of the testers, and we were required to ask them to ignore it and try and access the articles anyway. 8 of the testers were able to find the ‘Services’ link, however many commented that this wasn’t very clear, or that they would not expect to gain access through that tab. Only 4 of the testers clicked on ‘Services’ without prompting, and 2 testers became frustrated and did not complete the question. 1 tester accessed an article through an unexpected route – the links box in the full details screen. This was something that hadn’t been considered when writing the questions for the test. In further testing by library staff, it seems that the links box does not appear for all articles and as such cannot be considered a reliable link to full-text access.
Recommendations:
• Rename the ‘Services’ tab to something more intuitive – one tester suggested ‘check for full-text’ or ‘availability’. Another tester suggested using ‘Find it @ CU’ – we have re-named ‘Services’ as Find it @ CU – our SFX branding.
• Ensure that the availability line information is accurate – if this is not possible then turn it off completely to avoid misleading users – currently this is turned off and will be investigated again in the future.
• Do further testing on the links boxes.