Evaluating collections...

I've kept this posting from Stephen's Lighthouse "alive" in my Google Reader, even though I have read it several times. I've been intrigued mostly by the "honeycomb" of criteria that he actually discovered from a posting about User Experience Design:

Of course, the User Experience is the newest model to hit library assessment, so it was logical that Stephen then applied these criteria to evaluating online courses, asking questions pertinent to this library service.


I'm intrigued because I've been trying to develop my own model of evaluating collections that I could use on a regular basis.  I've been fixated on a "tripod" model, with 3 sets of 3 criteria - Usage, Scope and Depth - but I haven't made much progress down this path.  That is why this diagram caught my attention.  While  ascribed to online courses in this posting, these are the same kinds of criteria that have applied to collections of one kind or another.  

• Useful: Stephen asks, 

"What do your students do with the activities, materials, links and other content you include in your course? Does each of these itesm add to the understanding of a complex concept in some way? Or perhaps they directly support hte achievement of an established learning objective in the course?"

•  We could also ask, "What do your students (and faculty) do with the books, journals, databases, tutorials, and other content you offer in your library?"  
• Usable: Stephen focuses basic Web site usability issues, such as working links and applications.  And while we could also examine the usability of our Web interfaces to our collection (e.g. catalog, article databases, etc.), we could also evaluate the usability of our content (print books, ebooks, scores, etc.).
• Findable: While Stephen mentions "navigation and layout", findability of our resources is a very important aspect of collection assessment.  Are the items cataloged accurately and to the appropriate level? Are journal article findable using multiple access points (ejournals list, article databases, catalog, etc.)? What about the special collections?
• Credible: Stephen writes, "...it's important for the students to know the content is credible in terms of its source, purpose, currency and relevance" (original emphasis).  These are among the most established criteria of collection assessment.  The problem is how to measure them with some semblance of objectivity.  
• Desirable: "How well does your course capture the interest of your students?" Stephen asks.  Similarly, we could ask, "How well does our collection capture the interest of our students?"  While some of our faculty may snort at such a concern, if our students are not interested in the works, even for required projects and courses, they will eschew them or use them ineffectively.  
• Accessible: While Stephen's accessibility issues are focused on Web sites, we do need to consider other aspects, including physical accessibility.  Are book stacks wheelchair accessible? Even for those without overt physical limitations - are the book shelves too high? too low? Are the books packed in too tightly?  Do the special collections require too many people to "go through" to access?  While remote shelving is becoming more common, is the delivery too cumbersome to use?
• Valuable: Stephen asks, "Does your content support your students in their pursuit of course learning objectives?"  He quotes the author of the original article, "'the user experience must advance the mission'".  So, do our collections advance the mission of the library?

This may prove to be good model on which to base our regular collection assessment; or it may be simply the re-working of old ideas.  I need to ponder this more....

Friday Frags

I've been sidelined a bit this week viewing ALA's Virtual Conference, Mapping Transformation and preparing a presentation on DDA.  So this morning I've been playing catch-up with my blog reading and there are just too many interesting things to focus on one item of interest.  So here are my Friday Frags (tip o'the hat to our library's staff blog):

• Two items on Wikipedia:
• A study of how the copyright laws "impoverish" the transfer of knowledge, specifically through Wikipedia.  This is interesting because the economist uses a control group, so there is a little more validity than just descriptive stats.
• Conversely, there are much fewer contributors to Wikipedia, and more concerning is the drop of new administrators.  The article implicates the increased scrutiny that applicants are put under, with requirements of tests of copyright knowledge, essays, and reviews of past contributions and discussions regarding their input.  While this hasn't yet impacted overall stewardship, there is concern for the future of this community.  This would be a good microcosm of human behavior to study development and maturity of societies.
• The ALCTS awarded its Microgrants (sadly, I'm not a recipient), both of which were awarded different libraries of the University of Maryland.  The Health Sciences & Human Services Library will digitize early books and journals of medicine (prior to 1800), while the UM Baltimore County Library will collect, host and archive articles by faculty from open access journals that are endangered or no longer accessible.  
• The UK has been the source of several studies on the behavior of students and their interactions with libraries.  This study from JISC focused on doctoral students, most of whom could be considered within the "Generation Y".  Interesting findings include:
• Students are not participating in the scientific conversation as much as we would think.  Only 23% regularly review blogs, and only 13% actively participate, and only 9% maintain their own blogs.
• These students do not understand open access, believing that there is no positive and possibly negative value to making their own work available.
• Students predominantly rely on secondary sources, even those in the humanities.
• There is more, but it will take me a while to read through it all.
• Reports of other studies include:
• SAGE & LISU report on demonstrating library value
• Kno's report on usage patterns of e-textbooks by different student groups (lumped together by field of study).  It would be very interesting if this data could be replicated in the print world (but it would be very time-consuming and would lack the numbers for statistical analysis).
• Innovative digitization projects are cropping up, including:
• Digitizing the AIDS Quilt - not just for posterity, but also for research
• Interesting presentations & ideas related to the ALA Virtual Conference:
• Librarian by Day's post on the Needham & Williams' mantra of "Don't do more with less..." to libraries.
• Stephen Abrams' post on 3D printing - the idea of "making things" figured prominently at the Mapping Transformation conference.
• Also, Stephen Abrams' slide show: Mirages, Maps, Menus, Flowcharts & Dreams - I need to review this 'cuz I was attempting to multitask during the conference (a distinct problem of virtual conferences).

Unfortunately, that is all the time I have to write this post.  There is much more out there...

Library value & impact on learning

I've subscribed to the feeds from the Library Impact Data Project sponsored by JISC.  While I'm really interested in their results, my main interest is with their methods.  How will the researchers be able to make rational and scientific observations about library usage and academic outcomes that are valid and useful?  Well, the answer is, not easily, as this recent post explains.  The institutions in this project are all in Great Britain, which has more centralized data, such as "UCAS points" for getting into college, as well as demographic data.  In addition, the individual institutions require all students to log into the Library's resources individually, thus providing online usage for each student.  I wanted to find out what they could learn with such a relatively rich set of data.

The latest posting does suggest that even this level of data granularity is not enough.  There were problems in distinguishing "year of study" for each student (e.g. comparing usage for those in their first year versus in their second year, etc.).  This is important because usage does vary by year of study in and of itself, so it wouldn't be possible to control for this variation when examining the impact of other factors.  Their workarounds  (such as looking only at one year) effectively reduced their sample size to such an extent that statistical analysis of the results was not possible.  Finally, they had to convert one of their most key variables from a continuous (amount of time using the library) to a dichotomous variable (used/didn't use).  This, by their own admission, makes it more difficult to see the true relationship of usage to, say, completion rates.  


But given these "health warnings" of the data, they did share some intriguing results, notably that library usage is strongly and significantly associated with dropping out:

If you do not use the library, you are over seven times more likely to drop out of your degree: 7.19, to be precise. If you do not download PDFs, you are 7.89 times more likely to drop out of your degree.  Library PC usage also has a relationship with dropping out, although in this case not using the PCs makes you 2.82 times more likely to drop out of your degree.

While there may be some concerns about the levels of granularity of the measures, their strength should be indicative that there is some sort of relationship between usage and dropping out.  This, of course, should not surprise us considerably, and the results are certainly not counter-intuitive.  Furthermore, this is solely a correlation NOT an indication of causation - but I do like what is said about these results:

What it does offer, though, is a kind of ‘early warning system’. If your students aren’t using the library, it might be worth checking in with them to make sure everything is alright. Not using the library doesn’t automatically mean you’re at risk of dropping out – in fact, the number of students who don’t use the library and don’t drop out is much, much higher than the number who do leave their course prematurely. But library usage data could be another tool within the armoury of student support services, one part of a complex picture which helps them to understand which students might be at risk of failing to complete their degree.

This posting also demonstrates to me that even when with a much richer and granular data set, there will still be problems that through a monkey wrench into the best-laid plans.

Juxtapositions for today's high-minded professionals...

I've been noticing events, postings, and articles that center around the same basic theme - the battle of higher education against the bottom-line-focused executives that either administrate or serve on the boards of our institutions. Marshall Poe discusses the importance of the university press (a service closely tied to academic libraries), with some interesting insights into the problems these services face.  Then there is the UVa debacle with the summary dismissal of the president over "philosophical differences."  These differences were the result of the business-oriented board members' frustration with the "slowness" of the president's response to their concerns.

Those who pursued an academic career (like myself) claim that they purposefully avoided for-profit employment, sacrificing higher-pay and fancy perks, in return for both greater stability and a feeling of bettering the world.  Now that the corporate mentality has taken over politics, government, and organizational governance (even in the non-profit or social services world), we feel cheated. We have the worst of both worlds - lower-than-average pay for the same kind of work, greater demands on demonstrating outcomes, and greater risk of job loss.  This growing pressure also appears to be affecting the "warm fuzzies" we counted on getting from serving the youth, the growth of knowledge, the community (local and disciplinary).

While I do believe that, as custodians and beneficiaries of funding from taxes and gifts, we librarians have an obligation to be efficient and effective in our operations, I also firmly believe that governments and non-profit organizations cannot be run like a business.  A business' primary goal is to make money.  I do not deride that objective - I believe that capitalism based on fair-play is a far better economic model than government-run economies.  But if you look at the mission statements and goals of governments, non-profits, educational and other social services, making money is not mentioned.

Interestingly, I was reading a collection of classic papers and presentations in librarianship, one of which was from Ernest Cushing Richardson.  In his opinion piece published in the ALA Bulletin in 1927, he mentioned a "trend in librarianship in the present day" to "emphasize the method of efficiency, aptitude for action, the methods of modern salesmanship...over reflection, knowledge, learning, tact, sympathy, humane developments and other factors which go to set up a connection between the knowledge which is in books and the knowledge which lives in personality"1.  Richardson goes on to advocate that libraries are, indeed, a "real business which calls for business aptitude, business experience...".  He even points out the many aspects of library operations that have been adopted by business itself, including the use of the card file, the standardization of hte index card, and the idea of standards themselves.  However, he emphasizes that "the main thing about librarianship...[is] learning or knowledge," and that "the neglect of this aspect is even more fatal than the neglect of the business side."

So, it appears that this push-and-pull between running the library (and education) like a business and very foundations of connecting individuals to knowledge is not solely a 21st century concern. How can we, as both librarians and educators, work to ensure that we neglect neither the business nor the human or knowledge side?  Richardson advocates for a "philosophy of librarianship", one that gets to the "real nature and meaning of things."  This was a real problem to librarians in this time period, one for which I believe that Ranganathan made his greatest contribution.

Interestingly, Marshall Poe presents some very intriguing ideas about the decline of university presses.  While he does mention the influence of business-oriented approach to cutting anything that doesn't generate enough revenue, he also discusses the dependency of UP's on libraries to purchase these works (and the current decline in library budgets to continue this relationship), as well as the overall decline in reading in the population, even the learned population.  Finally, he blames the UP management themselves for not "getting" the Internet and how it could be leveraged to reduce the "brick and mortar" expenses and increase exposure and readership.  This, however, requires a major shift in a fundamental aspect of UP's: making content available for free.  This shift in focus from producing a product (a book) to providing a service (editing content provided by academics seeking recognition and promotion) could enable the presses to continue to serve their original mission of disseminating knowledge.

This appears to be one example of how the high-minded professionals in education and librarianship can manage the pressures to "economize" and change our practices and operations while still dealing in our core business of knowledge.

1. Richardson, Ernest Cushing. "Book and Person Who Knows the Book."  In, American Library Philosophy: An Anthology.  McCrimmon, Barbara, ed.  Hamden, CT: Shoe String Press, 1975; ppg. 51-62.

Redefining the Academic Library

Hmmm, I think that I've been tracking all the right blogs and reading all the right news related to library & information science, and there's still stuff that slips under the radar.  It's interesting the tracks information takes before reaching any one reader.  I imagine that if one were to visualize it, it would look like fractals.  Anyhow, I just now learned of the report from the University Leadership Council, Redefining the Academic Library: Managing the Migration to Digital Information Services. Released late last year, I must have missed the initial blog postings about it while I was shifting jobs back into librarianship.  It had the usual run, including mentions on Stephen's Lighthouse, LIS Trends, and Current Cites.  Interestingly, I could not find it on the Chronicle of Higher Education or Inside Higher Ed (is there a bias against the Education Advisory Board?).

Regardless, the title piqued my curiosity when I found it rather indirectly - a posting in LISNews about a series of blog postings in Attempted Elegance (Killing Fear) that form an essay regarding the recommendations of this report.  Sort of like a friend told two friends who told two friends...and so on and so on...

This report is intriguing, summarizing the information from various resources, many of which I was already aware of (e.g. "Value of Academic Libraries", the Taiga Forum, ITHAKA, etc.).  What this report provides is the distillation of this information into a cohesive strategy that libraries could take to ensure that they remain relevant. Essentially, this report elegantly supports Ranganathan's Fifth Law of Library Science: The Library is a Growing Organism.  An organism must grow in order to survive; and growth does not necessarily mean growth in size, but also evolution or ability to adapt.

Now, the authors admit that "(p)rognosticators have been warning of the disruptive capacity of computers, networks, and other information technologies for at least three decades, and predictions of the library's demise can be found as far back as the 1960's."  The authors believe that this time, there are four key drivers of change in libraries that are "converging and pushing more academic libraries toward a fundamentally different approach."  (Note: while the report is focused on academic libraries, these drivers apply to all libraries.)  These key drivers are:

• Unsustainable costs (particularly of serials)
• Viable alternatives to patrons (I would argue the level of viability, particularly for sources of scholarly information)
• Declining usage (hmmm, maybe of circulation and basic reference services, but not of online resources and in-depth reference services)
• New patron demands

As you can see, I have some questions about these drivers, but I understand the jist of their argument.  When asked with which sources our users (faculty or students) start their research, the most common answer is Google.  That makes sense; even librarians start with Google.  But that does not mean that the library is no longer viable.  Articles from non-OA journals are still the primary source of scholarly communication, all of which require money to access the content, at least within the first 6 months to a year.  Few scholars can afford personal subscriptions to all of their journals, nor all of their books (especially in the humanities & social sciences).  I do not deny the growing trend in non-traditional scholarly communication, but perhaps our end users are still not truly aware of the extent of resources that continue to be provided by the library.


I'm also skeptical of the claim of declining usage.  Foot traffic in most libraries has actually grown, and when you measure online usage of library-provided resources, then there is no evidence of declining usage.  Print circulation, in-house use of print materials, and basic reference services have, indeed, decreased quite noticeably.  We should take into account, however, the impact that our actions have had and will have on these trends.  For instance, by providing fewer books on the shelves for browsing and advertising the use of electronic books, should we not expect that circulation of print will decline further?


From the research, the report a list of 30 "lessons learned" organized around four themes:

• Leveraging digital collections
• Rethinking the scholarly publishing model
• Repurposing library space
• Redeploying library staff

These are not new ideas, but these lessons summarized as such provide some good talking points that librarians and library administrators can use to plan their strategic objectives and make their case for "redefining the academic library."  Another useful tool is the checklist, based on lessons learned, for "understanding your current practice."  This is, essentially, a list of practical and concrete steps that address the issues in the report. While there would be no guarantee that implementing all 39 recommendations would make your library relevant to your institution, it would be interesting to see if there is a relationship between these changes and indicators of relevancy (e.g. funding levels, overall usage, integration of the library throughout the institution).  

What do others think about this report?  Is it merely another one of its ilk that is read by administrators, touted in this year's strategic planning session, and then tossed in the file cabinet never to be read again?  Does it accurately represent the research on the future of academic libraries?  Is it unjustly biased against traditional libraries and print materials?  Is it useful?