Category: Development (page 1 of 2)

Discover Iterative Improvements for Tuesday, March 19

Please take note of the improvements coming to Discover during the Libraies’ IT maintenance window on the evening of Tuesday, March 19.

We have always known that it was important to clue users in to what type of results were displaying in each bento. The option to hover over the question mark graphic next to each label and view a description has been available. However, usability testing consistently demonstrated that users didn’t see the question mark as a call to action to hover over for them to get more information. In order to reduce cognitive load and clear up the confusion about what section would offer them the most appropriate results, we instead added a brief explainer next to each label.  

Explainers add to each bento

Red boxes added in screenshot for emphasis.

Another issue that bubbled to the surface from usability testing was in our Digital Collection bento. The default image icon being used when a thumbnail wasn’t available was seen by users as a clue that viewing that result would lead to an image, such as photo. But often, because the result would point users to a pdf of a document, the default icon when there is not thumbnail to display is now a pdf document icon:

Red box added in screenshot for emphasis.

The Applications Development and Support team continue to make improvements to the Articles+ segment.  The bento view results in Articles+ are currently arranged as scrolling tiles arranged horizontally.  While this works well for the Digital Collections segment, it didn’t work as effectively for this more text-heavy segment. 

Before: Horizontal scrolling

The new design will revert to a simplified vertical list view, showing three results, with a prompt to view more. 

After: Vertical List

To avoid confusion, the bento view is now also limited to peer-reviewed and full text results, just as it is in the focus view.  (See the 2/21 blog post on Discover improvements for more information about this.)    

And finally, to improve accessibility, we have improved the limiters that were displaying results in all lower-case text. Now, the capitalized results are easier to skim.  

Homebrew doesn’t work in Mojave (MacOS 10.14)?

If you upgraded MacOS to the new Mojave, you might have found that the Package Manager Homebrew no longer works correctly for some packages. If the same thing happens to you, where you get build errors, you might have to run the following command:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg  -target /

Give it a try next time you need to build a package with Homebrew.

Discovery Beta Refresh

Digital Initiatives and AD&S have been actively working hard on the discovery project as we work towards January 1st, 2019 as the target date to take the software out of beta.  We are excited about the progress being made and the plan that will take us into the new year. With this most recent update, you’ll see some exciting new concepts as well as the integration of a good deal of backend development designed to make the tool faster, easier to manage, and more consistent.

But first, Terry would like to thank some people for their continued hard work on the project… 

Stephen, our lead developer, has taken it upon himself to learn a variety of new skills and tools to really push the direction of this project.  Early on, we challenged him to find a better technical model that what was currently available – something that would be easy for us to manage and potentially easy for us to share.  And he’s delivered…incorporating modern web development techniques to develop a light-weight tool made for today’s internet and devices. 

We’d also like to highlight the UX partnership between AD&S and Digital Initiatives.  When we started the discovery project, we wanted to model a different type of development – one that put users at the center.  And this has at times been a challenge for the Libraries.  As we all know, it can be difficult to get a good representative sample when working with such a large population – so we had to look for partners.  Early this summer, we proactively reached out to the OSU student government, and they have been working in partnership with the Libraries to identify and provide students for testing.  It has been a fantastic partnership, and one that is giving us a much larger community to draw feedback.

Michelle has been regularly interviewing and working with our user community to understand some of the pain points in the new discovery tool.  In general, the feedback has fallen into a handful of specific categories:

  1. Confusion around the interface – while we have tried to reduce the reliance on library jargon (like facet), the many bentos, the scope of content…these are proving to be barriers. The many bentos are giving a lot of context, but ultimately, one of the challenges for students is a lack of understanding as to how we organize collections in the Libraries.
  2. Inconsistent links – we’ve been having some troubles with some interactions with the proxy and content returned via EDS. This hasn’t been completely resolved, but it should be soon.  However, this has lead us to over compensate on the interface by putting links everywhere.  Students are asking that when we link something (like an article title), that the results are consistent and reliable. To that end, we have reduced some unnecessary data and linking options.
  3. Simplifying Workflows – the process of getting to help or items is still taking too many clicks. We need to find ways to continue reducing the number of decisions made to get to valuable content.

Discovery Refresh

With this in mind, we spent a lot of time working on the back end to fix reliability and refresh issues, to address speed and performance concerns, and to take a hard look at how we present content to the Libraries.  So, with this refresh, you will  notice a few new things.  First, there is a new “view”.  While the default view continues to be bentos broken out by category, we have introduced a more integrated list view as a new option for Discovery. 

Discovery Refresh

This view has just the Articles+ and the All Library Content.  Users can toggle to this view by clicking the list icon in the upper left  of the application, and allows us to put all the content in the Libraries in context against the user’s search.  This option is provided specifically to address user feedback – to simplify finding content in the Libraries.   For users who still want to focus on a particular type of content,  the focus views continue to be available.  We’ll be performing usability testing this new interface and making a decision around the default view based on extensive feedback.

In addition to the new user interface option, we’ve implemented an updated indexing core and session management.  This will have two important impacts.

  1. It will allow us to develop more granular indexing rules around content types – enabling better discovery.  In our previous model, books shared the same indexing rules as EAD files.  In the new model, these can be different.  That’s enabling us to really push our indexing tooling.
  2. In the current discovery tool, sometimes pages would hang. This was sometimes related to how events (actions) occurred within different browsers.  To fix this, Stephen has implemented a shared session manager that will create a more reliable and faster user experience.

Going Forward

With these changes in place, AD&S and Digital Initiatives will be shifting Discovery development from feature development to quality development.  This means that over the next month, we will be specifically addressing user feedback – targeting pain-points and simplifying workflows.  This will include some new limits in Articles+ like limiting results to OSU owned content and peer reviewed materials, to a more straightforward process for users looking to find e-books or pass a search to OhioLINK.

Finally, if you are interested in providing feedback, Michelle is actively looking for feedback from the Libraries.  In addition to working with students, Michelle has done a number of usability sessions with faculty and staff in the Libraries.  If you want to be a part of that process, or know a student who might be interested, please let her know.

Running long processes without needing to be logged in

Ever need to index a large amount of objects overnight but don’t want to leave your computer in the office while is working?  Maybe you have a lot of commands you need to type and if your internet connection goes down, you have to start over from scratch?

Here is a quick and painless way to run a quick script without needing to be logged in while you’re waiting for it to finish.

First, VPN and log into the server you want to run a long process.

Then, enter the following command:

screen -R -D

Next, simply run the script that will take a really long time:

./this_script_will_take_many_days_to_finish.sh

Detach from the proccess with the following keystroke: “CTRL-A D

and logout:

exit

Check the progress

Anytime you want to check the progress of the script:

screen -R -D

And again, use “CTRL-A D” to detach from the process again.

There you go! If you have a long process or you simply want to be able to continue your work even if your internet connection dies, give this a shot!

Lint Your Code

In the grand scheme of things, writing code is easy. The challenge is writing clean code, to which there are many characteristics, from readability to conforming to best practices (e.g. principles such as SOLID).

One step you can take in writing cleaner code is using a linter, which is a tool that scans your code to ensure that it is free of syntactic and stylistic errors. Linters can potentially help you catch bugs before you even run tests against your code. Take them like a free second set of eyes – and yes, there are many free linters out there that you can use as a part of your workflow.

There are tools online that you can use, and certain text editors and IDEs that have some of this functionality built-in, but below are some editor-independent examples for some of the popular languages (and more) of today.

CSS: csslint
Docker: hadolint
HTML: HTMLHint
Javascript: eslint, jshint
JSON: jsonlint
PHP: phplint
Python: pycodestyle, pylint
Ruby: RuboCop
SCSS: scss-lint
Shell: ShellCheck
SQL: SQLint
XML: xmllint (part of libxml2)
YAML: yamllint

Discovery Project Update and Timelines

Submitted on behalf of Terry Reese: 

Over the past 3 months, Michelle, Russell, and I have had the opportunity to speak with many of you about the discovery project.  And I’d like to start by thanking everyone who took the time to attend the meetings, provide feedback, and help us better understand the user communities that you interact with and problems that many in our user and research communities face when navigating the Libraries resources.  This time was important for me as well, as it gave me an opportunity to talk about how the project is developing, the process that we are using to determine development priorities and talk about what it means to create a discovery tool that is more user-centered than our previous efforts, which I would argue have been more user-adjacent.  The work that the UX Cohort is doing around the library website redesign and the discovery testing will provide the libraries with a new and interesting model that should help inform much of our work going forward, and I’ve been grateful for their diligent work and the institution’s flexibility through this process.

Timelines

One of my goals with the discovery project and its implementation has been to minimize the disruptiveness of the process for the users and for instructors, while still valuing our agile design strategy of incremental delivery of functional improvement.  This will allow students and faculty an opportunity to transition to the new tools, and to allow faculty and instructors the time necessary to change various tutorials and instructional materials.  Our hope is that by staggering the releases across multiple terms, we can ease the transition for existing students and faculty, while providing a better discovery environment for this year’s incoming students.  I’m including a series of mock-ups that demonstrate how we plan to introduce the new Discovery tool and how we will slowly phase out our existing WorldCat Local search (our legacy tool).  It should be noted here that as we rearrange the search box and present users new options, the only specific tool that will ultimately be replaced is WorldCat Local.  Users (and librarians) will still be able to access the library catalog, OhioLINK, their favorite databases – directly.  The primary difference is that we’ll be collapsing the options that we provide on the home page in order to simplify and reduce the number of decisions that users need to make every day in order to find any content in the Libraries.  And through continuous user testing with the UX Cohort, we’ll continue to refine how discovery and resources are presented to our users.

The Timelines:

April 2018 – Jun 2018:  Initial beta on the new website

This will represent the first release of the discovery tool embedded into the Libraries preview website. 


Mockup (for illustrative purposes only)

Users will be able engage with the new tool through a new tab on the search bar, and it will be clearly marked that this resource is in beta.  At this point, what will be completed:

  • The tool will provide searching against our subscription content, subject guides (LibGuides), the Libraries’ websites and blogs, digital collections, the library catalog, and special collections resources.
  • The results will be linked to the beta website.
  • Integration with OSU Find It! and the Proxy will be completed.
  • Integration with OSUL chat services, and “escape hatches” will be available.

What will not yet be implemented:

  • Surfacing library expertise – this will not yet be completed as the bios work is still ongoing.
  • Transparent integration between Discovery, OhioLINK, and WorldCat – this work is planned for subsequent releases. We’ll be looking for partners as we think about this workflow for users.
  • Customizable query – we are specifically thinking about how discovery can be applied for our regional campuses and partners. This is work we will be evaluating over the Summer.
  • Specific database recommendations and integration with reference services (the current reference tab).
  • Final refinement of which subscription resources are included by default

June 2018 – Aug. 2018 (Out of Beta)

Over this period, the Discovery tab will move from the last tab, to the first, and the tool will become the primary method for querying content.  Users will still have access to all existing search services and tabs. 


Mockup (for illustrative purposes only)

New work completed:

  • Integration of workflows between discovery => OhioLINK => WorldCat
  • Initial implementation of expertise integration
  • Refinement of interaction design and user experience

Ongoing work:

  • Refining resource selection

What will not yet be implemented:

  • Customizable query – we are specifically thinking about how discovery can be applied for our regional campuses and partners. This is work we will be evaluating over the Summer.
  • Specific database recommendations and integration with reference services (the current reference tab).
  • Final refinement of which subscription resources are included by default

Aug. 2018 – Dec. 2018

Discovery project will be the primary search, but not the only available search.  Legacy (current) search options will continue to be available but will be collapsed into a single tab requiring users to select them for access (how this looks to users will need to be worked out). 


Mockup (for illustrative purposes only)

New work completed:

  • Ongoing refinement of which subscription resources are included by default
  • Specific database recommendations and integration with reference services (the current reference tab).
  • Customizable query – we are specifically thinking about how discovery can be applied for our regional campuses and partners. This is work we will be evaluating over the Summer.
  • Ad hoc working group put in place to prioritize and coordinate ongoing feature development of the discovery product.
  • Continued refinement of interaction design and user experience

Jan. 2019 –

Discovery replaces existing search tools, and WorldCat Local is turned off.

How do I get involved?

As you can see from the timeline, this is a big project with a lot of tasks.  Some of the future work will be prioritized in response to user feedback, while other tasks – like integration with OhioLINK and WorldCat – are elements that we know we must address to be successful and to provide a better experience for our users.  Additionally, I anticipate the way results are presented and options available will grow and morph as the UX Cohort works directly with users and librarians.  So, I anticipate a number avenues of formal engagement – specifically:

  • Related to the development of workflows that help users move transparently between our local resources, OhioLINK, and WorldCat
  • With Library Reference and Instruction – specifically as the organization plans for the Fall term and how this project will impact library instruction and engagement with public services
  • With our regional campuses, as we attempt to create a tool that is flexible enough to help solve their problems as well
  • OSU Mobile, and other campus communities that may be interested in embedding library content into their own environments

I’ve tried to indicate above where we anticipate working on some of these larger tasks, but if you are interested in taking up these conversations early – please let me know.

However, there is also a role for everyone.  The Discovery system will only get better the more users that we have working with it and providing feedback.  Up to this point, I realize that we still need to continue smoothing out the edges.  As we move the Discovery tool into our preview and public spaces, I would like to encourage users to start using  Discovery@OSU.  We realize that there will continue to be gaps, and that the tool won’t meet every type of user’s needs, but by helping us identify areas for improvement, we can create a better overall experience for our campus and research communities.  So, let us know if you find gaps or if there is something that is particularly useful.  Share with partners or patrons who are interested in such things and are interested in helping the Libraries make discovery easier for their communities.

The tool includes a feedback form and longer survey which can be accessed from the discovery tool results, but you can also submit information via a Hub ticket or at this point, while we are still in active development, to me, Michelle or Russell and we’ll follow-up so we can add the information into the project’s ticketing system. 

When you find something that might be interesting to us, please let us know as soon as possible, as we are changing configurations frequently.  Please include

  • what you were looking for,
  • the terms you were using,
  • what was returned,
  • what you expected to be returned, and
  • why that is important.

Wrapping up

As I mentioned early, I’m incredibly grateful for the time and feedback that many in the Libraries have provided throughout these early days of the discovery development.  It is my belief, that as we continue working on this project together, we’ll continue to move the libraries forward, and ultimately, provide a more enriching experience for our user and research communities.

–tr

Premier Plugin for ArchivesSpace

TL;DR

The Ohio State University Libraries releases first ArchivesSpace plugin to manage user defined fields.
______________

Just recently we’ve began to utilize ArchivesSpace as our archive management system. We went through a pretty thorough testing/review phase to ensure ArchivesSpace would fit the needs of the library. Throughout the entire process, things were documented so we could make an assessment on where we needed to immediately focus our development efforts.

There was a long list of requirements, and luckily, the majority of them could be addressed by making changes to configuration files or settings that already existed within ArchivesSpace. Sweet! However, there were a few that we knew would require a plugin – whether we would have to build it or not was to be determined. We came across a useful plugin called Generate Accession Identifiers which will do exactly as it sounds; generate a unique identifier when creating a new accession. This was great, but there was another requested feature that we couldn’t find an existing plugin for. So we built one!

This requested feature was to hide and/or alter the User Defined Fields (UDF). Now, labeling these fields could be done through some configuring of the en.yml, but, we could not find a way to hide the UDFs we weren’t using or simply did not want displayed. Since this feature was important to reduce confusion during the data entry process, we built our first small plugin for ArchivesSpace! We call it, Manage User Defined Fields. Clever, right? Anyhow, if you wish to check it out and perhaps use it for yourself, it’s up on GitHub at: https://github.com/osulibraries/manage_user_defined_fields. Pull requests are welcome!

DSpace Handle Server / Daemon – How to setup

We have just imported an existing DSpace Instance into Ohio State’s infrastructure, and one of the tricky steps was to reconfigure the DSpace Handle Server / Handle Daemon that ships with DSpace.

Step 1 – /dspace/bin/dspace make-handle-config

You have to follow the wizard /dspace/bin/dspace make-handle-config, and fill in details about your setup.
Continue reading

ElasticSearch start, stop, and restart

I use ElasticSearch, and I love it.
ElasticSearch bonsai tree
However, on my laptop, I frequently close and open my lid, thus suspending my system, and ElasticSearch doesn’t usually recover from that. So, I end up needing to kill ElasticSearch, and then starting it again. My old way was to hunt and peck to find the process, kill that, wait a moment, and then type an incredibly long command to start it again.

ps aux | grep ElasticSearch

peterdietz 25760 0.0 19.9 5969416 3330804 s002 S Sat07PM 13:12.16 /usr/bin/java -Xms256m -Xmx3g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=/Volumes/Data/elastic-testing/elasticsearch-0.19.11 -cp :/Volumes/Data/elastic-testing/elasticsearch-0.19.11/lib/elasticsearch-0.19.11.jar:/Volumes/Data/elastic-testing/elasticsearch-0.19.11/lib/*:/Volumes/Data/elastic-testing/elasticsearch-0.19.11/lib/sigar/* org.elasticsearch.bootstrap.ElasticSearch

kill -9 25760


/Volumes/Data/elastic-testing/elasticsearch-0.19.11/bin/elasticsearch

Thats no fun. So here’s the changes I made to find a better way. First, install the ElasticSearch Java Service Wrapper.

cd ~/Downloads
git clone https://github.com/elasticsearch/elasticsearch-servicewrapper.git
cd elasticsearch-servicewrapper
sudo ln -s /Volumes/Data/elastic-testing/elasticsearch-0.19.11 /usr/local/share/elasticsearch
sudo mv service /usr/local/share/elasticsearch/bin/

Then, at this point the ElasticSearch service wrapper is ready to go, you can test it by directly invoking the service:

/usr/local/share/elasticsearch/bin/service/elasticsearch

Usage: /usr/local/share/elasticsearch/bin/service/elasticsearch [ console | start | stop | restart | condrestart | status | install | remove | dump ]

Commands:
  console      Launch in the current console.
  start        Start in the background as a daemon process.
  stop         Stop if running as a daemon or in another console.
  restart      Stop if running and then start.
  condrestart  Restart only if already running.
  status       Query the current status.
  install      Install to start automatically when system boots.
  remove       Uninstall.
  dump         Request a Java thread dump if running.

So, pretty cool, I’ll go one step further, and shrink how much I’ll actually need to type. Edit your bash profile ~/.bash_profile and add the following.

alias es=elasticsearch

elasticsearch() {
  /usr/local/share/elasticsearch/bin/service/elasticsearch $1
}

Then you’ll need to re-source your bash profile.

. ~/.bash_profile

What this gives you is es is now mapped to a function that accepts an argument. The magic happens when you type:

es restart

Stopping ElasticSearch...
Waiting for ElasticSearch to exit...
Waiting for ElasticSearch to exit...
Stopped ElasticSearch.
Starting ElasticSearch...
Waiting for ElasticSearch...
.
.
.
running: PID:31782

So, now that your development process for interacting is that much more pleasant, or quicker, go forth to make more cool things.

Cloning an Elastic Search index to another server

As part of our development process for providing Usage Statistics of digital asset usage, we make heavy use of Elastic Search, a nosql data store, to handle storing and computing usage events from our logs. One of the tasks that became much easier today was to make a local development clone of our Elastic Search index.

ElasticSearch-head view of DSpace shard

Unlike traditional databases, there’s no dump-to-file button to click on, so I’m making use of a Node.JS tool that I’ve found at: https://github.com/mallocator/Elasticsearch-Exporter

Even though it is called an “Exporter”, its equally valid as an “Importer”, just reverse the flow of data.

// Move ElasticSearch data from A: production, to B: localhost

node exporter.js -a production-elastic-1.example.edu -b localhost

Let it run for a while, and eventually you start seeing messages like:

Processed 1066100 of 23643742 entries (5%)

Older posts