At Coventry University we have been implementing Primo (which we are branding as 'Locate'), Ex Libris's next generation Library Catalogue.
The main driver behind launching a next generation OPAC was to provide a single user interface for staff and students to use as the starting point for all their Library searches, no matter what type of content they are looking for. This will help provide a more consistent user experience and improve the efficiency of information retrieval. This should help to increase the use of the Library's e-resources. The new system should also help us to provide an attractive interface for our users to access quality information and we believe it will be a viable alternative to systems such as Google, Wikipedia, etc.
The output from our JISC Project will be a report into our implementation of Primo, which should be useful for other institutions considering launching a next generation OPAC and particularly for those deciding to implement Primo themselves. The report will also include an overview of the usability testing we carried out as part of our implementation. This testing was carried out with staff and students to try to identify ways in which we can make our Primo more intuitive and user-friendly. We have then made changes to the system based on these tests. We also gained further student input via a focus group where we discussed what is the key information to include on the Primo home page and how should this be laid out.
The general feedback we received about Primo was overwhelmingly positive, but we did identify issues which needed changes to be made, as they caused problems for several of the users involved in the testing. We have made numerous changes to the system already and we're waiting on Ex Libris to resolve some support issues before making other changes. We are also feeding back the resuts of our tests to Ex Libris and have been told that they will be taking these on board. We hope this means that where we discovered issues that cannot currently be changed Ex Libris will consider adding functionality that we require to future releases of Primo.
Implementation notes
Two IBM rack mounted servers [x3550 M2 8 core processing 16Gb RAM running Red Hat Linux 5.5 64 bit] were purchased and commissioned, one acting as a staging server, the other as the production server. In our case the supplier's software installation was carried out on the production server initially with a copy made to staging fairly late on in the project - normally this procedure is reversed. Project management was handled through the supplier's German office with conference calls established as and when required; these calls tended to increase in length and frequency the closer we moved towards our target switch-to-production (STP) date. On-site staff training was carried out from Ex Libris' UK office and was generally well received by staff.
Once the application had been installed, there were four main thrusts to the implementation work: the design of the end-user interface, taking on board the feedback received as part of the usability study; the testing of data export / import routines from our existing systems (Aleph, MetaLib, SFX and the University's repository product Equella); the establishing of automated data update routines and finally the installation / configuration of third party ancillary software products.
We were aware at the start of the project we had set ourselves an ambitious 'go live' date of early September, hoping to allow subject staff some time to prepare for the forthcoming student induction. In the event a decision has been taken to postpone the launch on two counts. Firstly Shibboleth configuration work is still incomplete and secondly Primo Central functionality is now scheduled to come on stream towards the end of October; Locate will be launched in the new year.
Our JISC project co-ordinator James Fisher leaves Coventry University on Friday 24 September 2010; we wish James well in his new post at the Univeristy of Warwick. Graham Daniels takes over as project contact following James' departure.
Tuesday, 21 September 2010
Monday, 6 September 2010
Focus Group - Primo Home Page
In order to get student feedback about what information should be on our Primo home page and the layout of the home page we arranged a focus group with ten students at the end of August. To get our participants we first emailed all of our volunteers who had given us their email addresses back in June. We had a much lower response rate than when we were arranging the usability tests. So to get more participants we went round the Library asking students if they would be available to come along to our group. We soon reached our target of ten students, and were very pleased that they all turned up at the appointed time.
The session was extremely productive with all the students contributing and lots of ideas being discussed. To start the session the three members of Library staff introduced themselves and then the ten students did the same and told the group what they were studying. To get everyone warmed up we asked them to tell us what they do first of all when they get an assignment and have to find relevant information. This produced a lively discussion and despite a number of students using Wikipedia and Google it was reassuring to find that everyone made use of the Library’s subscribed resources too and most seemed to use MetaLib to get to the databases. There was a quick debate around the pros and cons of Wikipedia. Most students agreed that they shouldn’t reference it and many had been given strict instructions about this by their lecturers. However, a number of students found it a useful starting point and used it to find search terms about their topic to enter into the Library databases. This was an interesting aside and seemed to get everyone involved from the very start of the session.
We then gave a quick overview of what to expect from the rest of the session. We also showed the group what the Coventry Primo currently looks like and did a quick demo of a search. We had some immediate positive feedback at this point about the look of the Primo interface. It was described by one user as “very easy on the eye.” Then we told them about the usability studies that had already taken place, and highlighted one of the changes that we had made because of these studies.
We then asked the participants to suggest as many ideas as they could think of that they would like to see on the home page. The list of suggestions that the group came up with and agreed would be useful on the home page was as follows:
· Announcements (e.g. work going on in the Library at the moment)
· Room booking / availability
· Search hints, tips / Search examples
· Introduction to Primo – Tutorial
· Contacts, Profiles, Who’s Who, Who’s responsible for certain subject areas
· Advertising for the University as a whole – Student Centre, Social Events, etc.
· Opening times
· Advanced Search
· Contact IT Services for help with your password, or with signing in
· Feedback section for Primo or for anything to do with the Library
· Ask a Librarian
· Latest books added to Library stock in different subject areas
· Passwords help, change password information
· Image of the Library (maybe as a watermark with links on top)
· A video with search hints
· Autocomplete functionality when typing searches
· Link to the Centre for Academic Writing & referencing information
· Library FAQs
· CU Portal link
· CUOnline / Moodle link
We then wrote each of the suggestions on post-its and worked on a flip-chart with a hand drawn version of the Primo home page. This had five areas on it. These were the header area which includes the search box, and the footer area. These two areas will remain fixed as users carry out searches. The rest of the home page is divided into three parts: a top left and top right section and a bottom section which goes across the whole page. The group suggested whereabouts on the home page was the best place to put each of their suggestions. There was quite a bit of debate around where was the best place to put each suggestion and some ideas were even rejected and removed from the list. In the end a consensus was reached with similar items being grouped together.
It was suggested that Announcements could appear in a ‘ticker’ format scrolling across the screen. This was something to look at in the future. There was a big debate around where on the page they should appear. It was agreed this should be limited to just Library announcements, and nothing else, and maybe just the top few announcements and headlines only to keep it brief. Eventually it was agreed that links to announcements could go on the top right of the page, along with new books added to stock and room bookings information.
Search hints should be at the top on the left under the search box. Also a short introduction to Primo should be here and possibly also a link to a video introduction, which is something else to look at in the future. An Ask a Librarian link should be in this area too. There was agreement to add a link to the text at the end of the search hints.
The Ask a Librarian link should also be in the header, so it is always available. The header and footer stay on the screen when a search is carried out, so would always be available, whereas the rest of the home page would be replaced by search results, etc. There was also consensus that a link to Library Passwords information should be in this section.
Contact information for Subject Librarians could go in the footer as this is a common place to find contacts. Links to IT Services & general ITS Passwords information could also go in this section, along with Library FAQs.
There was discussion over where to put the link to the student portal (CU Portal). Suggestions were to put this in the footer, but also to put a link to this at the top of the screen. Eventually we agreed to make the Coventry University logo at the top of the screen a link to CU Portal as this is the most likely place you would want to link to from Primo and also put another link in the footer.
It was suggested that the opening hours could go in the bottom section of the home page along with links to the Centre for Academic Writing and referencing.
The group thought it would be good if an image of the Library could be included on the page, possibly as a watermark. However, it was agreed that this should only be added if it didn’t distract users from the text on the home page.
We then looked at three other Primo Version 3 home pages, that are already live, and the group gave us input into what they liked and didn’t like about these sites.
After looking at the other sites it was agreed that the amount of text on the Coventry Primo home page should be kept to a minimum. One comment was “The simpler the better.” Ideally, the group wanted headings in bold that you could click on to make the information beneath those headers display. This would mean that initially there would just be headings on the page, which would prevent it from being too cluttered. However, it needs to be clear that this is what will happen if you click on the heading. The information also needs to be spread out enough to fill the screen.
The group also commented that the out-of-the-box image of books on a shelf, that displays on several Primo sites really drew the eye. However, this was probably the least important thing on the page. They still felt it could be good to include a watermarked image of our Library. However, we would need to be careful this didn’t detract from the information on the page if we included this.
Revisiting the mock-up of the home page it was again agreed to keep similar items grouped together but where possible to just display a general heading which you can click on in order to display the links beneath that heading. A picture of the final mock-up can be seen below:
One other point of interest that was raised several times during the session was the expectation of personalised information for individual users. Participants wanted information about previous searches, books that they have looked for or used before and new books in the Library relevant to their subject. We explained that much of this information is available in the area that we have named ‘My Favourites’. We didn’t go into too much more detail about this as we were focussing on the home page for all users, initially accessing it as a guest. However, it was clear that personalisation was something that should be looked at further in the future.
Overall it was an incredibly useful session. We will now sit down with the Library Systems Manager who is working on these kinds of developments in Primo and try to turn the comments into reality.
The session was extremely productive with all the students contributing and lots of ideas being discussed. To start the session the three members of Library staff introduced themselves and then the ten students did the same and told the group what they were studying. To get everyone warmed up we asked them to tell us what they do first of all when they get an assignment and have to find relevant information. This produced a lively discussion and despite a number of students using Wikipedia and Google it was reassuring to find that everyone made use of the Library’s subscribed resources too and most seemed to use MetaLib to get to the databases. There was a quick debate around the pros and cons of Wikipedia. Most students agreed that they shouldn’t reference it and many had been given strict instructions about this by their lecturers. However, a number of students found it a useful starting point and used it to find search terms about their topic to enter into the Library databases. This was an interesting aside and seemed to get everyone involved from the very start of the session.
We then gave a quick overview of what to expect from the rest of the session. We also showed the group what the Coventry Primo currently looks like and did a quick demo of a search. We had some immediate positive feedback at this point about the look of the Primo interface. It was described by one user as “very easy on the eye.” Then we told them about the usability studies that had already taken place, and highlighted one of the changes that we had made because of these studies.
We then asked the participants to suggest as many ideas as they could think of that they would like to see on the home page. The list of suggestions that the group came up with and agreed would be useful on the home page was as follows:
· Announcements (e.g. work going on in the Library at the moment)
· Room booking / availability
· Search hints, tips / Search examples
· Introduction to Primo – Tutorial
· Contacts, Profiles, Who’s Who, Who’s responsible for certain subject areas
· Advertising for the University as a whole – Student Centre, Social Events, etc.
· Opening times
· Advanced Search
· Contact IT Services for help with your password, or with signing in
· Feedback section for Primo or for anything to do with the Library
· Ask a Librarian
· Latest books added to Library stock in different subject areas
· Passwords help, change password information
· Image of the Library (maybe as a watermark with links on top)
· A video with search hints
· Autocomplete functionality when typing searches
· Link to the Centre for Academic Writing & referencing information
· Library FAQs
· CU Portal link
· CUOnline / Moodle link
We then wrote each of the suggestions on post-its and worked on a flip-chart with a hand drawn version of the Primo home page. This had five areas on it. These were the header area which includes the search box, and the footer area. These two areas will remain fixed as users carry out searches. The rest of the home page is divided into three parts: a top left and top right section and a bottom section which goes across the whole page. The group suggested whereabouts on the home page was the best place to put each of their suggestions. There was quite a bit of debate around where was the best place to put each suggestion and some ideas were even rejected and removed from the list. In the end a consensus was reached with similar items being grouped together.
It was suggested that Announcements could appear in a ‘ticker’ format scrolling across the screen. This was something to look at in the future. There was a big debate around where on the page they should appear. It was agreed this should be limited to just Library announcements, and nothing else, and maybe just the top few announcements and headlines only to keep it brief. Eventually it was agreed that links to announcements could go on the top right of the page, along with new books added to stock and room bookings information.
Search hints should be at the top on the left under the search box. Also a short introduction to Primo should be here and possibly also a link to a video introduction, which is something else to look at in the future. An Ask a Librarian link should be in this area too. There was agreement to add a link to the text at the end of the search hints.
The Ask a Librarian link should also be in the header, so it is always available. The header and footer stay on the screen when a search is carried out, so would always be available, whereas the rest of the home page would be replaced by search results, etc. There was also consensus that a link to Library Passwords information should be in this section.
Contact information for Subject Librarians could go in the footer as this is a common place to find contacts. Links to IT Services & general ITS Passwords information could also go in this section, along with Library FAQs.
There was discussion over where to put the link to the student portal (CU Portal). Suggestions were to put this in the footer, but also to put a link to this at the top of the screen. Eventually we agreed to make the Coventry University logo at the top of the screen a link to CU Portal as this is the most likely place you would want to link to from Primo and also put another link in the footer.
It was suggested that the opening hours could go in the bottom section of the home page along with links to the Centre for Academic Writing and referencing.
The group thought it would be good if an image of the Library could be included on the page, possibly as a watermark. However, it was agreed that this should only be added if it didn’t distract users from the text on the home page.
We then looked at three other Primo Version 3 home pages, that are already live, and the group gave us input into what they liked and didn’t like about these sites.
After looking at the other sites it was agreed that the amount of text on the Coventry Primo home page should be kept to a minimum. One comment was “The simpler the better.” Ideally, the group wanted headings in bold that you could click on to make the information beneath those headers display. This would mean that initially there would just be headings on the page, which would prevent it from being too cluttered. However, it needs to be clear that this is what will happen if you click on the heading. The information also needs to be spread out enough to fill the screen.
The group also commented that the out-of-the-box image of books on a shelf, that displays on several Primo sites really drew the eye. However, this was probably the least important thing on the page. They still felt it could be good to include a watermarked image of our Library. However, we would need to be careful this didn’t detract from the information on the page if we included this.
Revisiting the mock-up of the home page it was again agreed to keep similar items grouped together but where possible to just display a general heading which you can click on in order to display the links beneath that heading. A picture of the final mock-up can be seen below:
One other point of interest that was raised several times during the session was the expectation of personalised information for individual users. Participants wanted information about previous searches, books that they have looked for or used before and new books in the Library relevant to their subject. We explained that much of this information is available in the area that we have named ‘My Favourites’. We didn’t go into too much more detail about this as we were focussing on the home page for all users, initially accessing it as a guest. However, it was clear that personalisation was something that should be looked at further in the future.
Overall it was an incredibly useful session. We will now sit down with the Library Systems Manager who is working on these kinds of developments in Primo and try to turn the comments into reality.
Friday, 3 September 2010
Usability Testing Primo
About a month ago we carried out our usability tests on the out-of-the-box version of Primo. This had been set up to use data from our Aleph Library Management System test server. We also had data from MetaLib, our Federated Search Tool and SFX our Open URL Resolver harvested into Primo. We wanted to get input from staff and students on things that they found unintuitive, so that we could try to make improvements to the system before launching it.
Before starting testing we visited the University’s usability lab and talked things through with the technician. We also enlisted the help of an experienced usability tester, a PhD student from the University who had spent a lot of time running tests as part of her research. She gave us plenty of helpful advice and even attended our pilot study to give us feedback on ways to improve our test before running it with students and academic staff.
We ran the pilot study using a member of Library staff who had not seen our Primo installation at all, so had no preconceptions. Carrying out this pilot enabled us to familiarise ourselves with the environment in the usability lab and also to check that our questions all seemed to work and make sense to the tester. We did identify some changes to make to the questions before carrying out the testing for real.
We recruited 8 students and 2 members of academic staff to carry out the usability tests. We had obtained about 40 students’ contact details earlier in the project when carrying out a survey on what to call our Primo. This meant we were able to send emails to the volunteers with the incentive of paying for their time if they came in and worked with us. Although it was an unfortunate time to carry out the tests right in the middle of the summer vacation we still managed to recruit our 8 students without too many problems. Unsurprisingly we didn’t receive responses from a number of emails, but we were able to continue sending out requests until we had enough volunteers. Pleasingly, although we had to rearrange a couple of the tests at the request of the students everyone who volunteered did attend and arrived on time. The members of staff that we recruited were regular Library users who were suggested by the Subject Librarians who knew them.
We allowed an hour to carry out each test and to have a discussion afterwards and found that a number of different themes emerged, which helped us identify changes we wanted to make in order to make the system more user-friendly. Firstly there were things to do with the appearance of the interface, which we felt could be changed to make it more intuitive. For example, after carrying out a search the facets available to refine that search are displayed in a column on the left-hand-side of the screen. However, there is not much differentiation between these facets and the search results that display in the centre of the screen. Everything is shown on the same white background and there is only a faint line between the facets and the results. This meant that a number of our testers didn’t spot the facets for quite some time. Once they did spot them they did understand what they were for and used them intuitively to narrow down their search. We had a number of pleasing comments about how obvious it was what the facets did and also comparisons with Tesco Online and Amazon in the way that they worked. This showed how important it is to make the facets prominent to the users as they did find them useful once they noticed them. Therefore, in order to try to make them stand out more we added shading to the facets to differentiate them from the rest of the page and made the headings bolder. We also made the line between the facets and the results more defined. We believe this will make the facets easier to spot.
The second theme that emerged was to do with terminology that was used. An example of this is that the personalised area of Primo where users can save search results or previous searches, etc. was originally called the e-Shelf. We had a number of testers who thought that this would be where they would find e-books, which does have a certain logic to it: An e-book would be on an e-Shelf. Several other testers were also confused by the terminology and kept returning to this area, even though it wasn’t relevant to any of the tasks they have been set. Therefore, in order to try to avoid this confusion we have renamed the e-Shelf as ‘My Favourites’ as we believe this is clearer.
Another theme that emerged was to do with the functionality of Primo. For example when Primo finds multiple editions of an item it groups them all together under the Functional Requirements for Bibliographic Records (FRBR) rules. However, when searching by relevance it always prioritises the online version, even if this isn’t the most recent. In our experience with e-books, publishers often only make an electronic version of a title available some time after they have published the print version. This delay can be 6 months or it could be a year or more. This means that we often have new print editions of essential books available in the Library well before we are able to obtain an electronic version. We will often have previous print and electronic editions. This means that when Primo groups these records into a FRBR group the edition it chooses to display in the list of search results is the old electronic edition. The user won’t find out that we hold the most recent edition until they click on the FRBR group and examine the list of other available editions. When we asked Ex Libris if we could continue to search by relevance but make sure that the most recent edition of a book displays in the results list we were told this wasn’t standard functionality. Therefore, we had to raise a call to get our request investigated further and it’s possible that we won’t be able to make this change until a future version of Primo is released. We are waiting in hope for Ex Libris to investigate our request as we would like to make this change before we go live with Primo.
Most of the changes that we identified through the usability tests could be put into one or more of these categories. Since the tests both Paul here in the Library and Lee from Ex Libris have been working very hard to configure our Primo based on the results. It has already changed a great deal, but there is still more work to do. We have a consultancy day with Ex Libris here at Coventry next week, so we are hoping that we will be able to clear up many of the outstanding issues then.
Before starting testing we visited the University’s usability lab and talked things through with the technician. We also enlisted the help of an experienced usability tester, a PhD student from the University who had spent a lot of time running tests as part of her research. She gave us plenty of helpful advice and even attended our pilot study to give us feedback on ways to improve our test before running it with students and academic staff.
We ran the pilot study using a member of Library staff who had not seen our Primo installation at all, so had no preconceptions. Carrying out this pilot enabled us to familiarise ourselves with the environment in the usability lab and also to check that our questions all seemed to work and make sense to the tester. We did identify some changes to make to the questions before carrying out the testing for real.
We recruited 8 students and 2 members of academic staff to carry out the usability tests. We had obtained about 40 students’ contact details earlier in the project when carrying out a survey on what to call our Primo. This meant we were able to send emails to the volunteers with the incentive of paying for their time if they came in and worked with us. Although it was an unfortunate time to carry out the tests right in the middle of the summer vacation we still managed to recruit our 8 students without too many problems. Unsurprisingly we didn’t receive responses from a number of emails, but we were able to continue sending out requests until we had enough volunteers. Pleasingly, although we had to rearrange a couple of the tests at the request of the students everyone who volunteered did attend and arrived on time. The members of staff that we recruited were regular Library users who were suggested by the Subject Librarians who knew them.
We allowed an hour to carry out each test and to have a discussion afterwards and found that a number of different themes emerged, which helped us identify changes we wanted to make in order to make the system more user-friendly. Firstly there were things to do with the appearance of the interface, which we felt could be changed to make it more intuitive. For example, after carrying out a search the facets available to refine that search are displayed in a column on the left-hand-side of the screen. However, there is not much differentiation between these facets and the search results that display in the centre of the screen. Everything is shown on the same white background and there is only a faint line between the facets and the results. This meant that a number of our testers didn’t spot the facets for quite some time. Once they did spot them they did understand what they were for and used them intuitively to narrow down their search. We had a number of pleasing comments about how obvious it was what the facets did and also comparisons with Tesco Online and Amazon in the way that they worked. This showed how important it is to make the facets prominent to the users as they did find them useful once they noticed them. Therefore, in order to try to make them stand out more we added shading to the facets to differentiate them from the rest of the page and made the headings bolder. We also made the line between the facets and the results more defined. We believe this will make the facets easier to spot.
The second theme that emerged was to do with terminology that was used. An example of this is that the personalised area of Primo where users can save search results or previous searches, etc. was originally called the e-Shelf. We had a number of testers who thought that this would be where they would find e-books, which does have a certain logic to it: An e-book would be on an e-Shelf. Several other testers were also confused by the terminology and kept returning to this area, even though it wasn’t relevant to any of the tasks they have been set. Therefore, in order to try to avoid this confusion we have renamed the e-Shelf as ‘My Favourites’ as we believe this is clearer.
Another theme that emerged was to do with the functionality of Primo. For example when Primo finds multiple editions of an item it groups them all together under the Functional Requirements for Bibliographic Records (FRBR) rules. However, when searching by relevance it always prioritises the online version, even if this isn’t the most recent. In our experience with e-books, publishers often only make an electronic version of a title available some time after they have published the print version. This delay can be 6 months or it could be a year or more. This means that we often have new print editions of essential books available in the Library well before we are able to obtain an electronic version. We will often have previous print and electronic editions. This means that when Primo groups these records into a FRBR group the edition it chooses to display in the list of search results is the old electronic edition. The user won’t find out that we hold the most recent edition until they click on the FRBR group and examine the list of other available editions. When we asked Ex Libris if we could continue to search by relevance but make sure that the most recent edition of a book displays in the results list we were told this wasn’t standard functionality. Therefore, we had to raise a call to get our request investigated further and it’s possible that we won’t be able to make this change until a future version of Primo is released. We are waiting in hope for Ex Libris to investigate our request as we would like to make this change before we go live with Primo.
Most of the changes that we identified through the usability tests could be put into one or more of these categories. Since the tests both Paul here in the Library and Lee from Ex Libris have been working very hard to configure our Primo based on the results. It has already changed a great deal, but there is still more work to do. We have a consultancy day with Ex Libris here at Coventry next week, so we are hoping that we will be able to clear up many of the outstanding issues then.
Subscribe to:
Posts (Atom)
