Friday, August 31, 2012

More and more data on libraries

I finally had time to carefully examine the results from the Library Impact Data Project (sponsored by JISC). Just a reminder, this is a study that attempts to find the level, if any, of correlation between usage of library resources (as measured by visits, PC logins, logins to resources, etc.) and educational outcomes (as measured by grades, graduation level, and completion).  This is exactly what those who are associated with the ACRL Value of Academic Libraries project have been advocating.

Ellen Collins, the primary statistician for the study, posted the most statistically significant results to their blog (see Final Blog).  Unlike many statisticians, she has done an excellent job in presentation.  I was pleased that she posted effect sizes rather than raw values.  This is important because effect sizes are so much more informative than the values, even when the latter is presented with statistical significance.  Significance by itself only tells you that the differences between the groups you're comparing are not likely due to chance alone.  Sometimes this is because the differences are quite large - sometimes, particularly with very large sample sizes, the differences may be rather small.  By reporting the differences as "effect sizes", which takes into consideration variation within the groups, we, the readers, get a better understanding of how big that difference actually is.

It was also quite nice that she used colors to highlight the size and direction of the effect sizes.  This makes the charts so much quicker to read and easier to understand.  Here is one example:
Shows differences library usage by ethnic groups as compared to the majority (Whites).

Without this use of colors, she would have had to add more text to indicate the direction (the sign is irrelevant to direction) and it would have taken the reader more time to make sense of the values.  Finally, she does not include data for relationships that showed no statistically significant correlation.  This saves space and the time of the reader to skim through.  For a peer-reviewed journal article, though, she would likely need to show these values so that the reader would be able to verify this information.

Now to the results - Like all good statisticians, she starts with the basics - associations of demographics with the library usage. In her discussions, she is careful to detail some limitations to the results and analysis.  One of these is that some groupings were too small to analyze because they couldn't protect student identities and such analyses wouldn't produce valid conclusions.  Another is that the effect sizes of the associations of demographic variables with library usage, while statistically significant, were not very large (she thoughtfully included descriptions of ranges of effect sizes).  It's important to look at demographics first because these are factors that are beyond the control of the library, and if the effects were large, they could confound the relationship of associations between usage and outcomes.  I would have liked to have seen, however, similar analyses of demographics with educational outcomes.

After determining that demographics have very little effect on library usage, she then looked at disciplines in both broad and more narrow groupings (see Findings Post 2).  This has the most substantive, although admittedly less surprising, results.  Comparing all the other broad "subject groups" to social sciences (the "majority" set of users), those in the Arts, as well as Computer Science & Engineering fields used the library significantly less in nearly all measures of usage.  Interestingly, those in the Arts showed stronger effects of lower usage in the electronic resources measures.  Now, the Arts includes video and music, and there are a growing number of streaming services that would support these fields.  I glanced at the Huddersfield Library's resources for music and noticed only a few of these streaming services.  These results could provide the support needed for additional funding.  While it is not surprising that CS&E students did not borrow as many books from the library as those in the social sciences, our data of ebook usage shows that books these fields had greater usage than all the other STEM titles.  While we cannot determine who used these books, many of the titles are pretty specialized.

I'll have more about these results in another post...

Tuesday, August 28, 2012

This editorial provides some good advice for writing and submitting articles for publication in a peer-reviewed journal like CR&L. Key notes include:
  • Give the literature review section some meaning. Don't just list the studies that have been done before - provide "insights that highlight patterns, conflicts, or voids in the scholarship."
  • Have, and state clearly, a research question. The literature in library science is chock-full of "studies" that are merely descriptions of programs and outcomes. The authors equate having a methodology with having a research question - you can't have one without the other.
  • I'm surprised (and pleased) that the authors mentioned analysis & logic. I think this is a "lost art" that is no longer formally taught in graduate schools, let alone undergraduate, except in schools of philosophy.
  • Interesting comment about tone: "some writing is too informal and personal for presentation in a scholarly publication."
  • You'd think we librarians (many of whom are graduates of humanities education) would write with impecable grammar...apparently not.
  • Finally, a note about relevancy: "Some articles are
    well written and interesting but really have nothing new to contribute to the field."
However, this last issue can also serve as an obstacle to those wanting (or needing) to write and publish. I often hear of those who think that the ideas they have are not important to others - "Who would care?" This is where getting feedback from a trusted colleague or mentor is priceless.

In this editorial, the authors are actually referring to simple descriptions of personal experiences (single projects at their institution) that do not contribute to the growing body of knowledge.  Here are some tips that I believe would make our projects relevant:

      • Do a proper literature review (see comment above about giving the review meaning)
      • Have a formal research question
      • Develop a sound methodology to answer this question
      • Conduct the research in a responsible way
      • Discuss the results (even if the results are not "good" or supportive of your ideas) in a logical manner and within the context of previous knowledge.
      • Share your results in the most appropriate contexts.

Monday, August 20, 2012

Top trends of 2012 - a la ACRL

I had missed this when it had come out and had it not been for the posting in iLibrarian, I might not have seen it until, oh, 2013.  The ACRL Research Planning and Review Committee released its annual Top Ten Trends in Academic Libraries in the June, 2012 issue of College & Research Libraries.  I like these kinds of things because, when taken altogether, the most important issues tend to become more noticeable from the "noise" of the chaos of the information revolution.  I especially look for what I may have been missing.  So, here is their list and how I am involved, as Collection Assessment Librarian at the University of North Texas Libraries:

  • Communicating Value: "Academic libraries must prove the value they provide to the academic enterprise."
    • It is not terribly surprising that this issue is the first one listed.  It references the group's own Value of Academic Libraries report and the Lib-Value project.  While I'm not currently involved in value-related research, I know that it is being pursued by the administration and I hope to assess collection use with grades.
  • Data Curation: 
    • Data curation challenges are increasing as standards for all types of data continue to evolve; more repositories, many of them cloud-based, will emerge; librarians and other information workers will collaborate with their research communities to facilitate this process.
    • Our own Spenser Karelis and Shannon Stark have been pursuing this problem through action-oriented research.  I hope to eventually look into associations of data curation and availability with citation rates and grant funding.
  •  Digital Preservation:
    • As digital collections mature, concerns grow about the general lack of long-term planning for their preservation. No strategic leadership for establishing architecture, policy, or standards for creating, accessing, and preserving digital content is likely to emerge in the near term.
    • I recently read an article that reported on research on the impact of digital collections in history literature (Sinn, Donghee. 2012. Impact of digital archival collections on historical research. Journal of the American Society for Information Science and Technology 63 (8): 1521-37).  I would very much like to repeat this method evaluating the impact of our own digital collections on social sciences literature.
  • Higher Education
    • Higher education institutions are entering a period of flux, and potentially even turmoil. Trends to watch for are the rise of online instruction and degree programs, globalization, and an increased skepticism of the “return on investment” in a college degree.

      Shifts in the higher education surround will have an impact on libraries in terms of expectations for development of collections, delivery of collections and services for both old and new audiences, and in terms of how libraries continue to demonstrate value to parent institutions.
    • With the increasing pressure for libraries, particularly at publicly-funding institutions, to prove their value, collection development activities will likely include a wider array of assessment measures beyond simple usage.  I would like to develop a method to compare current and near-future curriculum and course planning with our collections in order to ensure that resources will meet the expected demand.  I would also like to compare highly-used resources against little-used resources to determine what could "predict" usage in the future.  
  • Information Technology
    • Technology continues to drive much of the futuristic thinking within academic libraries. 
    • This list emphasizes the issues and recommendations in the 2012 Horizon Report put out by The New Media Consortium.  These include social media/networks, collaborative learning, online/hybrid learning, and challenge-based and active learning.  While I wouldn't say that our library is at the bleeding edge of technological developments, I would say we have been early adopters of most technological trends.  I would, however, like to lead an assessment of how certain features and functions could be developed or modified to improve the findability, accessibility and usage of our collections.  For instance, I would like us to implement faceted searching in our catalog like we have in our digital collections (see this example).  In addition, I would like us to better integrate our ILLiad service with our online resources to make it easier to request physical items, especially as we have had to cut so many of our resources.
  • Mobile Environments
    • Mobile devices are changing the way information is delivered and accessed.
    • The UNT Libraries will be experimenting with loaning and access to resources for mobile devices. Like the item above, I hope to be involved in making collection decisions based, at least in part, on the accessibility of the content on mobile devices.  There is little point in paying for resources that have such poor usability that they are simply not used.
  • Patron-Driven E-Book Acquisitions
    • Patron-Driven Acquisition (PDA) of e-books is poised to become the norm. For this to occur, licensing options and models for library lending of e-books must become more sustainable.
    • This is something in which I am currently heavily involved.  Our own foray into PDA or Demand-Driven Acquisitions has been considered successful, with nearly 75% of ebooks purchased by our clients being used at least one time after purchase in the first 6 months of the project.  This far surpasses multiple usage of other ebook providers.  While I have analyzed the content of the collections fairly extensively, comparing subject distribution with our print and other ebook providers, I would next like to examine differences in usage based on more narrow subject categories, publishers, content-level, and publication year.  I would also like to learn how our patrons select and use these ebook resources, again, in an effort to be able to predict such usage.
  • Scholarly Communication
    • New scholarly communication and publishing models are developing at an ever-faster pace, requiring libraries to be actively involved or be left behind. 
    • The University of North Texas has been taking further strides in the paradigm shift of scholarly communication.  The university has recently required faculty to submit their scholarly outputs to its institutional repository (IR) (UNT Scholarly Works), and although there are certain allowable exceptions, it is significant step forward.  The Library has also stepped up efforts in this area by creating a position devoted to scholarly communication.  In addition, we have been making more concerted efforts towards supporting Open Access publications, by adding OA titles to our collection and providing OA publication fees for our faculty.  I would like to establish the regular review of the impact of our efforts, including OA articles, items in the IR and grant proposals.  I am also interested in the recent efforts to change the landscape of textbooks, particularly regarding OA and institutional licensing of texts for introductory texts.  
  • Staffing
    • Academic libraries must develop the staff needed to meet new challenges through creative approaches to hiring new personnel and deploying/retraining existing staff.
    • This is an issue that is of much concern at most libraries.  A balance of stability and flexibility is needed to ensure that our human resources are being used most effectively.  Simply moving people into new areas without appropriate training, consultation, or consideration of the human element will often result in failure.  Conversely, perpetual planning with little action will result in declining services and impact of the library.  I hope to be involved in the assessment of staff changes that could be used to make modifications and improve the effectiveness and efficiency of workflows.
  • User behaviors and expectations
    • Convenience affects all aspects of information seeking—the selection, accessibility, and use of sources.
    • The impact of the collection depends at least in part on the findability and usability of the resources.  I would like to learn how our patrons make decisions on which items to use for different purposes and to use this information to improve selection and findability.  
As you can see, most of my comments begin with "I would like to..." or "I expect to...".  These are aspirational in nature and reveal my ideas.  I just need to ensure that I follow through on these and not get sucked into routine or tangent efforts that distract me from what is more important.

Monday, August 6, 2012

"Library Cube"...on my wish list

In the article, Discovering the Impact of Library Use and Student Performance in EDUCAUSE Review Online, the Library Cube put together by the University of Wollongong Library in Australia is highlighted.  This data warehousing solution pulls together data about students (anonymized), schools (called, "faculties"), and library resource usage.  The library administrators and librarians who manage programs and collections can not only demonstrate that those who use the libraries resources score higher grades, but also can identify groups that do not use the resources as much as others.  This could be used to investigate reasons and develop methods and programs to address this disparity.

As Collection Assessment Librarian, here is what I could do with this data:
  • Associate resource usage (print and online) with grades, degree completion, schools, years in the program, on- versus off-campus, student backgrounds (like locations of high schools or community colleges, family income, work loads, ages)...
  • Associate collection coverage with courses much more easily than the manual or semi-manual process we do now.
  • Demonstrate to faculty any disparities of usage of resources assigned (this refers to the copyright case in Georgia where it was found that they were not in violation of copyright for many of the items posted online because they were simply not used!)
  • Determine how different or how similar different student groups are in terms of library usage.
  • Identify the groups of students or faculty who never use the library in order to conduct additional research to find out what they do use and how we could reach them better.
This kind of connection of student and school measures with resource usage measures is becoming more prominent in other countries, but less so here in the United States.  

This line from the video, "At the heart of Library Cube is the Student Number," points out the key reason, I believe.  Concerns about privacy have led to policies that unnecessarily restrict the sharing of key data.  There are ways to make the data effectively anonymous, but there is still great reluctance.  Technology is another obstacle, but one that should not be hard to overcome.  Our library is veering ever so slightly in this direction, starting with our face-to-face instruction sessions and some courses that are taught largely through Blackboard.  I hope that the results will convince the decision-makers that this method can and should be used to help make informed decisions while not infringing on students' rights and not imposing unnecessary work on the data gatherers.

Wednesday, August 1, 2012

Textbook affordability & libraries

With the economic crisis droning on ad infinitum, I've decided that I really missed out not taking an economics course.  When planning my liberal studies program, I had avoided anything that was even close to business courses.  Having lived an additional 20 years, though, I have realized that economics is as important a subject in the social sciences as anthropology and history.  So, I've picked up a few books at my library here (I love being at a general university!) and have started learning a few things.  For instance, did you know that as resources become limited, the cheaper goods become more in demand?  OK, I had learned that one on my own (why else would I shop at Walmart?), but having these basics now at the front of my mind increases my awareness of the economics of everyday life.

Thus it is with textbooks.  As funding for education gets tighter (for students and for institutions), we start looking around for cheaper alternatives.  Textbooks are third in the list of educational expenses - behind tuition and room & board.  My concerns about textbooks has grown since I returned to the field of librarianship and sold many of my old public health books.  Three years before, I had gotten rid of my old LIS books (many of which dated to the 1980s & early 1990's).  What was the point?  I kept these books because they were expensive and because I thought I would need to refer to them regularly in my professional life.  This was not the case.  Except for my biostats book (Rosner, 6th edition - great book), I've rarely re-opened any of my texts.

Some of the best classes I took did not even include a textbook - the readings were hand-selected by the professor and the discussions drawn from them were stimulating and enlightening.  This has led me to conclude that textbooks are a shortcut for faculty whose time is limited for teaching.  This is effectively pushing the cost of developing a curriculum from the institution (who should have paid for the time of the faculty to create the curriculum) to the student and/or the financial aid providers.

So it is with interest that I've been reading about alternatives to textbooks (necessity is the mother of invention, eh?).  Here's a posting on the ACRL Value of Academic Libraries blog about the Alternative Textbooks project. It's exciting to see libraries involved in getting faculty to use its resources (what a thought!) and helping student-retention.  The University of Southern Florida's T.A.P. site serves as a good hub of information about this paradigm shift in curriculum planning. The biggest initiatives towards making textbooks more affordable include Open Access textbooks and class licensing of e-textbooks (McGraw-Hill), which may be a compromise with publishers.

The programs mentioned in the ACRL Value posting are more intriguing because of their involvement with the libraries.  California State University system advocates for faculty to work with their library to identify local resources to support the curriculum.  The key to these programs is providing small amounts of money to pay the faculty to develop their own "texts" using OA and library resources (emphasis added):
At Temple University Libraries we are now in the second year of our local Alternate Textbook Project. In each round ten faculty receive a $1,000 grant to develop an alternate textbook. Although boosting student retention is not explicitly stated, the Project has multiple outcomes: (1) save students money to improve college affordability; (2) encourage faculty to make more use of library and open educational resources; (3) improve student learning.
I'm very interested to see how these projects pan-out and how they could enable libraries to directly impact student retention and add value to students' and universities' outcomes.