<word of warning: this is a shameless plug for my library>
I had neglected to mention in my first post about TLA that the UNT Libraries' Portal to Texas History was awarded the TLA Wayne Williams Library Project of the Year. This the Digital Projects' "crown jewel" of the Digital Collections, with a significant amount of grant funding and effort provided for its development. This collection of primary resources has become integrated in the public schools' curriculum, particularly for its Texas history requirement. It is definitely worthy of the award.
In the second general session, UNT Libraries' again was honored with not only the Best in Show, but also the PR Plan Winner and the Collateral Materials Winner honors of the TLA Branding Iron Awards. This is the result of a recent focus on marketing, advertising, and external relations. The UNT Libraries takes advantage of the artistic and creative talents of the UNT's students, as well as innovative librarians and hard-working staff, to develop marketing plans and advertising materials in a wide variety of formats. One popular product is the subject liaisons posters, complete with photo, placed strategically in the right places of the stacks.
</end shameless plug>
Now, as they were awarding the Upstart Innovative Programming Award to Eileen Lee of the Montgomery County Memorial Library System for her Sensory Storytime for developmentally disabled children, I had the idea of collaborating with MLS faculty and our UNT Autism Center to develop collections and services for autism spectrum children. Not exactly totally original, but I wanted to get my idea down on paper before I forgot it.
I had not paid enough attention to the program about the speaker for the second general session - in fact, I had considered skipping out after the awards and getting some coffee (my one complaint of the TLA facilities - not enough coffee!). I'm glad I stayed. I've been enjoying the segments with Dan Aierly's on NPR, and when I realized who the speaker was, I knew I wouldn't need the coffee. His focus was on his latest research on cheating - essentially, most of us cheat a little. Interestingly, providing simple reminders before a test about the "honor code" (even if none exists) decreased the number of people who cheated. Well, on to the sessions of the day, only two of which will I write about.
Library Assessment Today: This was an overview of the experiences of Jim Self of the University of Virginia. He's been involved in library since the early 1980's, long before the current "fad" of assessment. But he starts with a quote from J.T. Gerould of the Princeton libraries from 1906, stating that the basic questions of assessment were essentially, "Is this method the best? Are we up to the standard of a similar institution?" So clearly, assessment is not a new fad. It is a traditional aspect of librarianship - we've just started looking further away from library collections and outputs, to the broader interests of the institutions we serve. Furthermore, as Jim states, recently, the user has become the center of the discussions.
Interestingly, Jim mentioned that they had conducted their own patron satisfaction surveys every three to four years since 1993. They did participate in LibQual survey in 2006, but was disappointed by the much lower response rates. They returned to their own customized surveys with a steady 50% response rate. They have seen their patron satisfaction increase over time. He attributes a notable "U" turn in the satisfaction with the online catalog with an attempt to "re-invent" the system. They have evaluated the use of the collection and redefined collection development, with a focus on the user. Now, they are focusing on "budget communications" with university administration. Like most academic libraries (well, all libraries), there has been a dramatic decrease in the number of reference questions asked. But, we both asked, is that necessarily a bad thing? Jim suggests that this may be due, at least in part, to librarians doing a better job with our systems. He summarizes his first section with a listing of how libraries could provide better evidence of our worthiness, including usability testing, "wayfinding" (evaluation of our facilities), ethnographic studies (a la Rochester), quantitative performance measures (aka "balanced scorecard"), MINES, COUNTER & e-resource use, and staff surveys.
Jim then switches to observations from the consultation work he has provided to other libraries over the last five years. He has seen more libraries accept assessment as a core activity, but that it is still hard to sustain. He has also seen more collaboration with individuals and groups both on- and off-campus. But he emphasizes that what is still needed are common measures of holdings, usage, costs and learning outcomes, as well as sharing of this information. He would also like to see more standard survey templates and metrics, and perhaps an "Index of Performance".
Mr. Self ends his session with some observations on value and assessment in general. The current need is to determine the library's impact on student and faculty success, but this may be difficult to measure. There are natural limits of assessment, in that we are attempting to predict the future based on our past and present, and goodness knows, that often turns out to be wrong. He pointedly noted that "innovation does not come from assessment, but it can indicate what works". Local barriers to assessment include complacency, fear, arrogance, inertia, an operations mindset, and ethical concerns.
Finally, he asks about the the point of view of assessment -- is it neutral and unbiased? Or is it advocacy? Would assessment ever result in recommendations for a lesser library? Perhaps, but the purpose of the our work is to improve the service - to give users what they need when they need it.
Right Size, Right Stuff: This was part two of Fort Worth Public Library's "market segmentation" presentations. This was more of the nitty-gritty of this method - applying the data to actually modify a library's collections. Actually, the market segmentation was only part of this process. The bulk of the work was adjusting collection size based on usage. Like the Dallas Public Library, the FWPL has shifted to a "floating collection", with each branch's collection set by what patrons return. So while they will delivery materials to a branch for an individual, once the patron returns the material to the branch, it stays there - until requested elsewhere. This saves quite a bit of money by not having to return materials. But it also leads to a "pooling effect" in which the collection size of certain heavily-used branches grows dramatically. So the collection development librarians for the FWPL System charted the current holdings against the usage to determine the "right size" for each collection at each branch. The collections were broken down by age level (adult, juvenile), type (fiction, picture book, etc.) and subject. Here is a sample table that was developed for a single branch:
<I'll add this later>
Essentially, the "right size" is calculated as such: % of Total System Use X Total Titles in System. This was then used to calculate shelving space (using different formulas for different collections). They also used market segmentation to make additional adjustments to the collection based on interests and hobbies.
Actually, I was quite intrigued by this use of market data. I'm not sure how, if at all, such info could be used in an academic setting - the data is pretty well set on households. But I was inspired to think outside my little box.
Tuesday, April 30, 2013
Notes from the 2013 Texas Library Association Annual Conference
The annual conference of the largest state-based library association, boasting over 7,000 members (CLA claims only 3,000+ members and NYLA simply describe their membership at "several thousand") took place in Fort Worth ("Cowtown") this year. I took advantage of the close proximity and attended three of the 4 days of the main conference (I had to miss Saturday's sessions to care for a sick dog, who, I believe, was just pretending so that I would take him out to the park). The Texas Library Association is strongly supported by public and school librarians, and academic and special librarians tend to feel not so much excluded as simply overwhelmed at the conferences. We academic librarians do participate in the ALA-sponsored ACRL, so it's not like we have no outlet for our professional growth. However, it is important that all librarians be included in the issues of librarianship that are most relevant to our state. This is particularly true given the changes in education politics over the last 10-20 years. So here are my notes of sessions that were of most interest to me.
General Session I: The most memorable moment of the first general session was the awarding of the Librarian of the Year to Lydia Tucker, school librarian in San Antonio. Based on the statement of the award, she certainly seemed to have deserved the award. And based on her reaction, she certainly did not expect it.
Transforming Libraries for Engagement, Gary Strong, UCLA: In this session, Gary Strong, University Librarian at UCLA, detailed the progress his library has made in, well, transforming for engagement. He compared the libraries when he started there as director as a 7-11 store - "people get their quart of milk and leave"; there was little engagement. He described how the UCLA libraries have become "participatory libraries", with space, both physical and virtual, for teaching, learning and research. They did this by engaging faculty and students with new forms of teaching and learning, making student research more visible, and encouraging interdisciplinarity. Notably, the UCLA libraries have embrace alternative roles, including a lab, museum, gallery, performance space, and civic meeting site. My observation, however, is that libraries have always played these roles - some more than others. Maybe it's only been the last 10-20 years that libraries have reduced these roles in an effort to focus on collections and reference.
One very interesting insight Gary brought up was that many students come from areas with few library resources and engagements. This may become more and more common if we cannot convince school boards and city councils that libraries have a substantial ROI on their primary constituents. Thus, it should not be expected for them to expect the services that we do provide. Rather, we need to reach out to these students to prove what we can provide for them. He then describes several new pedagogical approaches, including:
General Session I: The most memorable moment of the first general session was the awarding of the Librarian of the Year to Lydia Tucker, school librarian in San Antonio. Based on the statement of the award, she certainly seemed to have deserved the award. And based on her reaction, she certainly did not expect it.
Transforming Libraries for Engagement, Gary Strong, UCLA: In this session, Gary Strong, University Librarian at UCLA, detailed the progress his library has made in, well, transforming for engagement. He compared the libraries when he started there as director as a 7-11 store - "people get their quart of milk and leave"; there was little engagement. He described how the UCLA libraries have become "participatory libraries", with space, both physical and virtual, for teaching, learning and research. They did this by engaging faculty and students with new forms of teaching and learning, making student research more visible, and encouraging interdisciplinarity. Notably, the UCLA libraries have embrace alternative roles, including a lab, museum, gallery, performance space, and civic meeting site. My observation, however, is that libraries have always played these roles - some more than others. Maybe it's only been the last 10-20 years that libraries have reduced these roles in an effort to focus on collections and reference.
One very interesting insight Gary brought up was that many students come from areas with few library resources and engagements. This may become more and more common if we cannot convince school boards and city councils that libraries have a substantial ROI on their primary constituents. Thus, it should not be expected for them to expect the services that we do provide. Rather, we need to reach out to these students to prove what we can provide for them. He then describes several new pedagogical approaches, including:
- Inquiry-based learning & Inquiry Labs
- Peer-to-Peer Learning (as an alterative to classes)
- Game-based Learning (this is becoming more and more common)
- Partnering with innovative faculty
Finally, Gary goes into details about several of the programs and services that effectively transform the libraries for engagement. Most prominent was the Library Research Commons, which is lab space that supports the whole life-cycle of research. Of importance is that the space is meant to encourage discussion, not simply experimentation. The space can be reserved for use by faculty and/or students, but this reserved usage requires collaboration with librarians. This ensures that the libraries are involved in the projects and can bring the full breadth of resources and knowledge. When not reserved, the space is available for informal use by anybody. They have been, however, overwhelmed by interested in formal projects and are now having to prioritize projects.
Gary also points out the work in software development, notably for mobile apps, which are developed by students who are hired as software developers. While they are managed by a permanent staff member, it is their ideas that are developed, tested and implemented. Essentially, they are expected to "create stuff (they) will use."
The one final point that I wanted to emphasize from Gary's presentation was that rather than embedded librarianship, they are focusing on embedding faculty in the library. This ensures that teaching, learning and research are intertwined with library resources and services.
LibValue: Paula Kaufman, of the University of Illinois at Urbana/Champaign, described their role in the LibValue project, a 3 year project that is not yet completed. The first phase looked at how the library is involved in the faculty grant process. Through surveys asking faculty about the role library resources played in their grant applications, they were able to estimate a $4.38 ROI. This is based on the percentage of faculty who rated citations in proposals as important to the percentage of proposals. The second phase conducted the same ROI assessment globally to determine variations based on geography and institution type.
The bulk of the presentation was on the third phase, which involved multiple institutions and organizations, including ARL, UIUC, University of Tennessee, Stanford and JISC. Through a series of surveys, they determined the value of library collections in teaching, reading and scholarship. Some of the preliminary results show that fewer percentage of students preferred using e-books than faculty..In addition, measures of usage the digital special collections were developed and tested. Tools for determining ROI for academic libraries will be available from the ARL LibValue site.
Course and Subject Guides Usage: It was an unexpected pleasure to attend the presentation by Amy Pajewski, from West Texas A&M (WTAMU). It was short but sweet, with some interesting ideas about assessing subject guides. While her study was closer to usability study than qualitative research, she did learn what at least some students think about the subject guides that librarians hold so near & dear to our hearts. Not much. Really. The students thought the guides were too busy ("stressed" and "overwhelmed" were key words given), could not tell to whom the guides would be useful, and when the guides should be used. Looking at some basic usability guidelines and research, Amy could immediately see some gross violations of good design.
Then she asked some librarians some key questions about guides & their development. Of particular importance was that only 2% of the librarians asked had talked with faculty and/or students before developing their guides. While this may be bad design, it is not terribly surprising. After all, faculty & students are busy folks. Also, it is hard to get users to express their future information needs effectively. However, some good qualitative research should be able to discern the information needed for common scenarios (beginning a paper, researching a grant, etc.). I also wonder how much, if any, of the knowledge gained from information behavior research that has been conducted has been incorporated into guide design.
Getting a read on your borrowers: This session, conducted jointly by Debra Duke of the Fort Worth Public Library and Chris Briggs of Bruxton Analytics was the most interesting session I went to all day. Over the last year or two, FWPL had contracted with Bruxton to provide market segmentation analysis of the users of their branch libraries. Using techniques developed for commercial organizations, they were able to combine demographic data and circulation patterns of their patrons with market-level data about their populations likes, dislikes, work habits, hobbies, interests, political bents, and consumer spending patterns. This was all part of the library's "20/20 Vision Master Plan". Through this analysis, they determined that the average drive time for active users to their nearest library was 8 minutes, with 2-15 minute range. But they noticed that the spread of users of individual branches was quite wide, with some users preferring branches farther away, at least from their homes. They also used the market segmentation data to get a "read" on their non-users. Of particular importance were the hobbies, reading preferences, and interests.
Debra emphasized that a major obstacle to initiating this work was, as she put it, the "ick factor". That is, the distaste of librarians to share potentially confidential information about their patrons with a third party. This was partly overcome by the contract stipulations, as well as the company's reputation and experience dealing with companies & other organizations with similar concerns. What they haven't yet decided to do is take that information a step further and actually customize advertising material for potential users based on the market segmentation data. This would be based more on the market research data and not what the patrons used in the libraries, but it does give one pause; should we walk into this environment of less and less privacy? Or should we stubbornly stand as a vanguard, even if others no longer expect it?
Well, that was just day one. I'll post my notes of day two separately. Don't worry - that's all there is, since my dog's feigning illness kept me home Saturday.
LibValue: Paula Kaufman, of the University of Illinois at Urbana/Champaign, described their role in the LibValue project, a 3 year project that is not yet completed. The first phase looked at how the library is involved in the faculty grant process. Through surveys asking faculty about the role library resources played in their grant applications, they were able to estimate a $4.38 ROI. This is based on the percentage of faculty who rated citations in proposals as important to the percentage of proposals. The second phase conducted the same ROI assessment globally to determine variations based on geography and institution type.
The bulk of the presentation was on the third phase, which involved multiple institutions and organizations, including ARL, UIUC, University of Tennessee, Stanford and JISC. Through a series of surveys, they determined the value of library collections in teaching, reading and scholarship. Some of the preliminary results show that fewer percentage of students preferred using e-books than faculty..In addition, measures of usage the digital special collections were developed and tested. Tools for determining ROI for academic libraries will be available from the ARL LibValue site.
Course and Subject Guides Usage: It was an unexpected pleasure to attend the presentation by Amy Pajewski, from West Texas A&M (WTAMU). It was short but sweet, with some interesting ideas about assessing subject guides. While her study was closer to usability study than qualitative research, she did learn what at least some students think about the subject guides that librarians hold so near & dear to our hearts. Not much. Really. The students thought the guides were too busy ("stressed" and "overwhelmed" were key words given), could not tell to whom the guides would be useful, and when the guides should be used. Looking at some basic usability guidelines and research, Amy could immediately see some gross violations of good design.
Then she asked some librarians some key questions about guides & their development. Of particular importance was that only 2% of the librarians asked had talked with faculty and/or students before developing their guides. While this may be bad design, it is not terribly surprising. After all, faculty & students are busy folks. Also, it is hard to get users to express their future information needs effectively. However, some good qualitative research should be able to discern the information needed for common scenarios (beginning a paper, researching a grant, etc.). I also wonder how much, if any, of the knowledge gained from information behavior research that has been conducted has been incorporated into guide design.
Getting a read on your borrowers: This session, conducted jointly by Debra Duke of the Fort Worth Public Library and Chris Briggs of Bruxton Analytics was the most interesting session I went to all day. Over the last year or two, FWPL had contracted with Bruxton to provide market segmentation analysis of the users of their branch libraries. Using techniques developed for commercial organizations, they were able to combine demographic data and circulation patterns of their patrons with market-level data about their populations likes, dislikes, work habits, hobbies, interests, political bents, and consumer spending patterns. This was all part of the library's "20/20 Vision Master Plan". Through this analysis, they determined that the average drive time for active users to their nearest library was 8 minutes, with 2-15 minute range. But they noticed that the spread of users of individual branches was quite wide, with some users preferring branches farther away, at least from their homes. They also used the market segmentation data to get a "read" on their non-users. Of particular importance were the hobbies, reading preferences, and interests.
Debra emphasized that a major obstacle to initiating this work was, as she put it, the "ick factor". That is, the distaste of librarians to share potentially confidential information about their patrons with a third party. This was partly overcome by the contract stipulations, as well as the company's reputation and experience dealing with companies & other organizations with similar concerns. What they haven't yet decided to do is take that information a step further and actually customize advertising material for potential users based on the market segmentation data. This would be based more on the market research data and not what the patrons used in the libraries, but it does give one pause; should we walk into this environment of less and less privacy? Or should we stubbornly stand as a vanguard, even if others no longer expect it?
Well, that was just day one. I'll post my notes of day two separately. Don't worry - that's all there is, since my dog's feigning illness kept me home Saturday.
Saturday, April 13, 2013
Feedback on the latest ITHAKA survey
Last week, ITHAKA, the non-profit "that helps the academic community use digital technologies to preserve the scholarly record and to advance research and teaching in sustainable ways," released the results of its latest iteration of a wide-reaching survey of faculty about "tracking the attitudes and practices of
faculty members at US colleges and universities for more than a decade" (pg. 5). You can download the report and read for yourself how faculty attitudes and practices have changed. I'll be reporting, however, the reactions of others to these results.
The first reaction I read was actually in an email from a colleague - "Doesn't really inspire too much enthusiasm…", which itself was none-too-inspiring. Essentially, attitudes of faculty towards the library haven't changed notably over time - the majority of faculty who responded still view libraries primarily as buying agents, gateways, and archives. Depressingly, fewer than in 2009 agreed with the statement that the library helps their productivity in research & scholarship and that the library supports their teaching activities. Finally, less than 60% believed the library helps undergraduates develop critical thinking, research, and information literacy skills. <sigh>
Even more non-inspiring was the breakdown by broad disciplines - as expected, faculty in the humanities were most supportive of all of the library's roles asked in the survey. And as suspected, faculty in the sciences primarily see libraries as a buying agent. Fewer than half of science faculty considered the library important in their research or teaching activities, particularly (and depressingly) the latter. Of course, faculty in the social sciences responded in between these extremes of "love" and "disinterest". But only about half of them thought that the library supports their teaching activities. Hrrmph.
Barbara Fister, in Library Babel Fish, is similarly uninspired by the results, although she brings up some interesting insights. Given the lack of consideration of the library's role in supporting the development of critical thinking and information literacy skills, she suggests that maybe libraries redirect their attention:
Barbara brings up a point, though, that casts doubt on the validity of the report - the incredibly low response rate of 3.5%. Of course, this 3.5% of the over 160,000 recipients to whom invitations were sent results in 5,261 respondents. The question is, then, are the 5,261 representative of the sample population. Once the data are deposited in the ICSPR, this can be determined.
Others, including Steven Harris of Collection=Connection, noted the vast disparity of opinions about the library's roles between library directors and other faculty. This "disconnect" is apparent in the nearly 40% differences in responses of agreement to statements about the library's roles in supporting teaching activities and research productivity. Harris suggests it may be a matter of lack of faculty awareness...maybe it's something else?
While Rick Anderson in the Scholarly Kitchen mentions the "less surprising results" (increasing use of search engines by faculty, faculty don't see the library as important as library directors, etc.), he does point out results that he found "more surprising". These include a modest increase in the use of the catalog as the initial search tool (although I, like others, think this is an anomaly), and a "modest" decrease in comfort with the shift from print to online journals.
Carol Straumsheim looks at the use of technology in the classroom, or lack thereof. A posting in the blog, Scholarly Workflow, focuses on the the issues of, well, scholarly workflows. The poster notes the low percentages of faculty who value institutional support of research and publication process in such ways as providing Web pages of research outputs, assessing the impact of work, determining the most optimal venues for publication, etc. The ranges of support were all below 50%, indicating that most faculty appear to prefer to go it alone.
Overall, the reception of the 2012 ITHAKA S+R Faculty Survey has been modest and less than enthusiastic. Generally, the faculty have their own opinions about the value of technology and libraries, and they are very different from librarians. How can we bridge this gap? More food for thought...
faculty members at US colleges and universities for more than a decade" (pg. 5). You can download the report and read for yourself how faculty attitudes and practices have changed. I'll be reporting, however, the reactions of others to these results.
The first reaction I read was actually in an email from a colleague - "Doesn't really inspire too much enthusiasm…", which itself was none-too-inspiring. Essentially, attitudes of faculty towards the library haven't changed notably over time - the majority of faculty who responded still view libraries primarily as buying agents, gateways, and archives. Depressingly, fewer than in 2009 agreed with the statement that the library helps their productivity in research & scholarship and that the library supports their teaching activities. Finally, less than 60% believed the library helps undergraduates develop critical thinking, research, and information literacy skills. <sigh>
Even more non-inspiring was the breakdown by broad disciplines - as expected, faculty in the humanities were most supportive of all of the library's roles asked in the survey. And as suspected, faculty in the sciences primarily see libraries as a buying agent. Fewer than half of science faculty considered the library important in their research or teaching activities, particularly (and depressingly) the latter. Of course, faculty in the social sciences responded in between these extremes of "love" and "disinterest". But only about half of them thought that the library supports their teaching activities. Hrrmph.
Barbara Fister, in Library Babel Fish, is similarly uninspired by the results, although she brings up some interesting insights. Given the lack of consideration of the library's role in supporting the development of critical thinking and information literacy skills, she suggests that maybe libraries redirect their attention:
William Badke notes in his comment to this post that information literacy is not on the radar of higher education, with few journals including articles on IL, citing a publisher whose reason for rejecting a manuscript was that there was "'no market for information literacy books among educators'". A quick search of our discovery tool, Summon, and Google Scholar for peer-reviewed articles on "information literacy" and "higher education". About 1/5th of the results were from higher-ed journals (e.g. College Teaching and Studies in Higher Education), and a smattering communications and educational technology journals (notably EDUCAUSE), but about 75% were from library journals. A quick search of WorldCat retrieved over 400 titles (not limited by publication year), but many of the initial results were from traditional librarianship publishers (Neal-Schuman, Libraries Unlimited, ALA, Scarecrow Press, etc.). So Badke's comment does bear out, at least on first glance.(This makes me think we should stop writing so many articles about information literacy for other librarians and think about reaching the faculty. Just a thought.)
Barbara brings up a point, though, that casts doubt on the validity of the report - the incredibly low response rate of 3.5%. Of course, this 3.5% of the over 160,000 recipients to whom invitations were sent results in 5,261 respondents. The question is, then, are the 5,261 representative of the sample population. Once the data are deposited in the ICSPR, this can be determined.
Others, including Steven Harris of Collection=Connection, noted the vast disparity of opinions about the library's roles between library directors and other faculty. This "disconnect" is apparent in the nearly 40% differences in responses of agreement to statements about the library's roles in supporting teaching activities and research productivity. Harris suggests it may be a matter of lack of faculty awareness...maybe it's something else?
While Rick Anderson in the Scholarly Kitchen mentions the "less surprising results" (increasing use of search engines by faculty, faculty don't see the library as important as library directors, etc.), he does point out results that he found "more surprising". These include a modest increase in the use of the catalog as the initial search tool (although I, like others, think this is an anomaly), and a "modest" decrease in comfort with the shift from print to online journals.
Carol Straumsheim looks at the use of technology in the classroom, or lack thereof. A posting in the blog, Scholarly Workflow, focuses on the the issues of, well, scholarly workflows. The poster notes the low percentages of faculty who value institutional support of research and publication process in such ways as providing Web pages of research outputs, assessing the impact of work, determining the most optimal venues for publication, etc. The ranges of support were all below 50%, indicating that most faculty appear to prefer to go it alone.
Overall, the reception of the 2012 ITHAKA S+R Faculty Survey has been modest and less than enthusiastic. Generally, the faculty have their own opinions about the value of technology and libraries, and they are very different from librarians. How can we bridge this gap? More food for thought...
Saturday, April 6, 2013
Providing support for research & publication
A paper by two of my colleagues has just been released in pre-publication of College & Research Libraries (C&RL), one of the top library science journals, particularly for academic libraries. The article, Fostering Research and Publication in Academic Libraries, presents results of a survey my colleagues had been working on for the last two years. I like this article for more reasons than the fact that I know & respect the authors...I like that it was replicating research completed 35 years ago, and thus providing good comparisons over time. This is so rarely done in social science research (and even in natural sciences). Replicating research is important to verify findings, validate instruments, and provide data that can be compared over time. So I'm proud of the authors for doing the work and proud of C&RL for publishing (one of the biggest reasons replicating research is not done is because it is usually not published).
I also like the purpose of the article, which was to get a sense of the support that is being provided to librarians who are expected to publish research. This is important because "not all academic librarians are
prepared to meet these requirements because of time constraints and a lack of training." Think about it...teaching and research faculty in the social sciences have to complete at least 60 more hours of classes, including research methods and statistics, conduct some kind of research project, write a dissertation or several articles, and then either support the dissertation or submit the articles to the peer review process. Few librarians go through a program that provides that same experience.
I appreciated my education at Texas Woman's University in the mid- to late-1990's, where we were required to take a research methods course and a social sciences statistics course. We did have the choice of completing a research project or a professional project, but we had to write a "professional paper" - which, for those of us who opted for the research, was on the level of a master's thesis. Then we had to participate in a poster session where we had to support the project to the dean himself (yes, I was nervous when he stopped by my poster). My understanding among my peer librarians is that this was not a common educational experience.
So what kind of support do the 70% of ARL deans (who responded to the survey) who expect their librarians to research and publish provide? They say that their libraries provide "library work time for research and publication" - although a few do not support that. Yet so often I hear from my colleagues (of this and other institutions) that they just don't have the time. There is a definite dissonance between what institutions say is provided and what librarians perceive is allowed. Maybe it's the amount of time that's allowed versus what is thought to be needed. The median amount of time provided is 1-5 hours per week (that's a big spread: 2.5% - 12.5% of a 40-hour work week). Interestingly, about 40% of the responding libraries provided 6-10 hours (up to 25%). Most libraries also offer sabbaticals, but the percentage of pay provided is down from 30 years ago.
What about funding? Well, nearly all library deans say they support internal funding for research projects, with most of them actually providing money, usually from the library's discretionary budget. Is it enough money? Well, the amount provided has increased by 50% in the last thirty years - is that enough? A quick look at the inflation calculator shows that the total inflation rate between 1980 and 2012 was 191%.
While most deans provide support for training and informal mentoring for research activities, only about half provide formal mentoring programs for their librarians. And even fewer provide "library research committees", defined by the authors as groups "organized to support the research and writing activities of librarians." Only about a third of the responding deans indicated that such activities were provided at their libraries. And what's more, the gap between what was supported by the deans and what was actually provided was notable for the formal training and mentoring activities.
The vast majority of responding deans provide their libraries with "project support" - as in computers, photocopying, mail, even financial incentives for survey or focus groups. Less than half, however, provide support for student assistants. This is down from 30 years ago.
OK, so what does all of this mean? What did my peers conclude? Well, more ARL libraries provide faculty status for their librarians and, concomitantly, require their librarians to conduct research and publish for promotion. While the stated dollar amounts of internal funding have increased, it did not keep up with inflation. More deans state that they think it is right to provide support for research and publication, but fewer state that their libraries actually do provide it.
Time seems to be the biggest concern for librarians. Yet, as mentioned by the article's authors, this was so even for a university that provided upwards of 20% of work time for research activities, as well as sabbaticals. I'm very interested in this phenomenom. My colleagues in this article mention other comparisons of work schedules and division of tasks between academic librarians and other academic faculty. They particularly point out that faculty have been found to have variations in the divisions of their time within the year and between years, depending on their research activities. So maybe the issue is that librarians do not schedule these variations themselves; instead, they continue to provide their services year-round, continuously which leaves only 5-10 hours a week. And not much can be accomplished with this kind of scheduling. Rather, librarians should consider taking "sabbaticals" from their regular duties (akin to academic faculty who have a lighter teaching load one year) to focus 50-75% of their time to a research project. Can this be done without affecting quality of library services?
I think that the main support feature of interest to the authors, though, was the "library research committees," if only because that is what has been attempted at our library. We have a larger committee that attempts to coordinate training and support groups aimed at different foci of research. Some groups are more active than others, and evaluation of their success is difficult to measure. I lead a small group, and I know that it is hard to find the balance between support and pestering, finding just the right amount of pressure to apply to each person. I think the reason these groups have a hard time catching on and producing measurable results is that they require excellent leadership, which is hard to find, and the right mix of people, which can vary year to year.
Overall, this was a very good article and I'm interested to see what others will do with the information. Will deans realize that they need to do more than say they think support should be provided? Will other libraries attempt some kind of support group or formal mentoring program? Will librarians change the way they devote time to their research activities?
Of course, the whole issue of whether research & publication should even be expected of practicing librarians is for another time...
I also like the purpose of the article, which was to get a sense of the support that is being provided to librarians who are expected to publish research. This is important because "not all academic librarians are
prepared to meet these requirements because of time constraints and a lack of training." Think about it...teaching and research faculty in the social sciences have to complete at least 60 more hours of classes, including research methods and statistics, conduct some kind of research project, write a dissertation or several articles, and then either support the dissertation or submit the articles to the peer review process. Few librarians go through a program that provides that same experience.
I appreciated my education at Texas Woman's University in the mid- to late-1990's, where we were required to take a research methods course and a social sciences statistics course. We did have the choice of completing a research project or a professional project, but we had to write a "professional paper" - which, for those of us who opted for the research, was on the level of a master's thesis. Then we had to participate in a poster session where we had to support the project to the dean himself (yes, I was nervous when he stopped by my poster). My understanding among my peer librarians is that this was not a common educational experience.
So what kind of support do the 70% of ARL deans (who responded to the survey) who expect their librarians to research and publish provide? They say that their libraries provide "library work time for research and publication" - although a few do not support that. Yet so often I hear from my colleagues (of this and other institutions) that they just don't have the time. There is a definite dissonance between what institutions say is provided and what librarians perceive is allowed. Maybe it's the amount of time that's allowed versus what is thought to be needed. The median amount of time provided is 1-5 hours per week (that's a big spread: 2.5% - 12.5% of a 40-hour work week). Interestingly, about 40% of the responding libraries provided 6-10 hours (up to 25%). Most libraries also offer sabbaticals, but the percentage of pay provided is down from 30 years ago.
What about funding? Well, nearly all library deans say they support internal funding for research projects, with most of them actually providing money, usually from the library's discretionary budget. Is it enough money? Well, the amount provided has increased by 50% in the last thirty years - is that enough? A quick look at the inflation calculator shows that the total inflation rate between 1980 and 2012 was 191%.
While most deans provide support for training and informal mentoring for research activities, only about half provide formal mentoring programs for their librarians. And even fewer provide "library research committees", defined by the authors as groups "organized to support the research and writing activities of librarians." Only about a third of the responding deans indicated that such activities were provided at their libraries. And what's more, the gap between what was supported by the deans and what was actually provided was notable for the formal training and mentoring activities.
The vast majority of responding deans provide their libraries with "project support" - as in computers, photocopying, mail, even financial incentives for survey or focus groups. Less than half, however, provide support for student assistants. This is down from 30 years ago.
OK, so what does all of this mean? What did my peers conclude? Well, more ARL libraries provide faculty status for their librarians and, concomitantly, require their librarians to conduct research and publish for promotion. While the stated dollar amounts of internal funding have increased, it did not keep up with inflation. More deans state that they think it is right to provide support for research and publication, but fewer state that their libraries actually do provide it.
Time seems to be the biggest concern for librarians. Yet, as mentioned by the article's authors, this was so even for a university that provided upwards of 20% of work time for research activities, as well as sabbaticals. I'm very interested in this phenomenom. My colleagues in this article mention other comparisons of work schedules and division of tasks between academic librarians and other academic faculty. They particularly point out that faculty have been found to have variations in the divisions of their time within the year and between years, depending on their research activities. So maybe the issue is that librarians do not schedule these variations themselves; instead, they continue to provide their services year-round, continuously which leaves only 5-10 hours a week. And not much can be accomplished with this kind of scheduling. Rather, librarians should consider taking "sabbaticals" from their regular duties (akin to academic faculty who have a lighter teaching load one year) to focus 50-75% of their time to a research project. Can this be done without affecting quality of library services?
I think that the main support feature of interest to the authors, though, was the "library research committees," if only because that is what has been attempted at our library. We have a larger committee that attempts to coordinate training and support groups aimed at different foci of research. Some groups are more active than others, and evaluation of their success is difficult to measure. I lead a small group, and I know that it is hard to find the balance between support and pestering, finding just the right amount of pressure to apply to each person. I think the reason these groups have a hard time catching on and producing measurable results is that they require excellent leadership, which is hard to find, and the right mix of people, which can vary year to year.
Overall, this was a very good article and I'm interested to see what others will do with the information. Will deans realize that they need to do more than say they think support should be provided? Will other libraries attempt some kind of support group or formal mentoring program? Will librarians change the way they devote time to their research activities?
Of course, the whole issue of whether research & publication should even be expected of practicing librarians is for another time...
Wednesday, April 3, 2013
Dealing with the aftermath of cancellations
Due to factors beyond the immediate control of our library, we had to reduce our materials budget for this current fiscal year by about one-seventh. This was my initiating into collections management, essentially cutting journals and databases. For journals, key factors involved in making decisions on what to cut included:
- Usage
- Length of embargo from aggregator
- Cost (both total cost and cost-per-use)
Although we had worked with our subject liaisons and had communicated the cuts to the faculty through both the liaisons and our dean, it has only been since January when they've really taken notice. This is because of the inevitable delays between release of the final budget and actual elimination of service. One such incident provides a good example.
We had decided to cancel the subscription to Proceedings of the National Academies of Science (PNAS) primary because our $3000+ subscription fee was paying for only six months worth of content, and about 20-25% of these articles were Open Access. The copyright fee for PNAS is $2.00 - a quick glance at the articles in PNAS (research articles) suggests an average article length of about 8-10 pages. It would take, then, about 150 ILL requests to make up for the cost (OK, less than that if you factor in indirect costs of ILL, but you get the point). Indeed, there had only been one ILL request for a PNAS article this year.
Yet, our liaisons are hit by (admittedly a vocal few) irate faculty who are indignant that we would not have a current subscription. I hope the liaisons inform these reasonable people of the factors involved. I wish we could include this information in a Notes field in the MARC record -- maybe they'd read it, maybe not.
Sunday, March 17, 2013
Research misconduct in LIS?
The editorial, Bogus Evidence by R. Laval Hunsucker in the latest Evidence-Based Library & Information Practice (EBLIP), discusses at great length the potential for research misconduct in the LIS field. After setting the stage with the recent apparent rise in "questionable research practices" (QRP's) and outright fraud in the basic, medical, and social sciences, Laval brings our attention to our own profession, or rather, the lack of attention that our profession has given this issue. He questions whether we, as members of the LIS profession, should consider research misconduct to be more or less or equally prevalent than in other research fields. He admits that there is enough evidence to support any of these positions, yet not enough evidence to reach any conclusions. And that, Laval asserts, is the crux of the problem. Why should we assume that we are any better (or worse) than any other field? And if we are neither better nor worse, shouldn't we be concerned that we are equally bad?
Laval brings up some very valid points, particularly regarding the difficulty of detecting research fraud. Indeed, having been involved in a few research studies, I can imagine where fraud could occur, if so desired, particularly with data. Auditing the data collected is rarely done, and yet, it is, I believe, the weakest (or easiest) point. I have heard of large surveys where one single survey-taker fraudulently completed forms. Proper follow-ups detected the problem, but not before so many had been submitted that the integrity of the entire study had been threatened. But that is an easier problem to detect because the researchers were themselves conducted the study with integrity. The more difficult cases occur when the researchers manufacture the data. Only a full data audit could detect this, but as mentioned above, this is so rarely done because it is difficult and time-consuming (and thus, costs money).
So, like so many researchers, those conducting studies in LIS are trusted to collect, analyze, and report their data in an unbiased and appropriate manner with little oversight. Is this trust justified? It appears that in this day and age of competition for jobs, promotions, and respect, people are growing more deceitful. But could we not say, too, that in this day and age of growing transparency with the Internet, people are growing more skeptical and distrustful?
Professionally, I'm less concerned with outright fraud in the LIS literature than with QRP's related to poor training and limited knowledge. This is particularly true regarding studies conducted by practicing librarians like myself, rather than LIS research faculty who have completed more formal training and apprenticeship in research, in the form of the dissertation. Most practicing physicians do not initiate and formally conduct clinical studies. There are many who participate in research, but very few actually develop proposals, gather the data, analyze the results, and write the papers of their own research. Yet academic librarians are very often expected to do so themselves, often with less training than the physicians receive. Therefore, why should we not expect QRP's to occur? Laval himself notes that the "good news, and the other important difference, is that genuinely fraudulent research in LIS is almost certainly far less prevalent than sloppy research in LIS."
Actually, the apparent increase in research misconduct does need more study in order to establish the environment in which our research can be trusted. But to judge all studies with a jaundiced eye will make it more difficult to move ideas into practice. Laval discusses the many proposed solutions, ranging from changes to rewards and incentives to formalized research integrity training. Like any complex problem with multiple foundations, no solution addressing just one of these foundations will work by itself. But like they tell those with mental or addiction problems, simply recognizing the problem exists is the first step.
Laval brings up some very valid points, particularly regarding the difficulty of detecting research fraud. Indeed, having been involved in a few research studies, I can imagine where fraud could occur, if so desired, particularly with data. Auditing the data collected is rarely done, and yet, it is, I believe, the weakest (or easiest) point. I have heard of large surveys where one single survey-taker fraudulently completed forms. Proper follow-ups detected the problem, but not before so many had been submitted that the integrity of the entire study had been threatened. But that is an easier problem to detect because the researchers were themselves conducted the study with integrity. The more difficult cases occur when the researchers manufacture the data. Only a full data audit could detect this, but as mentioned above, this is so rarely done because it is difficult and time-consuming (and thus, costs money).
So, like so many researchers, those conducting studies in LIS are trusted to collect, analyze, and report their data in an unbiased and appropriate manner with little oversight. Is this trust justified? It appears that in this day and age of competition for jobs, promotions, and respect, people are growing more deceitful. But could we not say, too, that in this day and age of growing transparency with the Internet, people are growing more skeptical and distrustful?
Professionally, I'm less concerned with outright fraud in the LIS literature than with QRP's related to poor training and limited knowledge. This is particularly true regarding studies conducted by practicing librarians like myself, rather than LIS research faculty who have completed more formal training and apprenticeship in research, in the form of the dissertation. Most practicing physicians do not initiate and formally conduct clinical studies. There are many who participate in research, but very few actually develop proposals, gather the data, analyze the results, and write the papers of their own research. Yet academic librarians are very often expected to do so themselves, often with less training than the physicians receive. Therefore, why should we not expect QRP's to occur? Laval himself notes that the "good news, and the other important difference, is that genuinely fraudulent research in LIS is almost certainly far less prevalent than sloppy research in LIS."
Actually, the apparent increase in research misconduct does need more study in order to establish the environment in which our research can be trusted. But to judge all studies with a jaundiced eye will make it more difficult to move ideas into practice. Laval discusses the many proposed solutions, ranging from changes to rewards and incentives to formalized research integrity training. Like any complex problem with multiple foundations, no solution addressing just one of these foundations will work by itself. But like they tell those with mental or addiction problems, simply recognizing the problem exists is the first step.
Saturday, March 2, 2013
New results from impact studies
The ACRL Value of Academic Libraries site highlighted two articles recently published from the University of Minnesota, led by Shane Nackerud. Both articles are published in the April 2013 issue of portal: Libraries and the Academy, although I'll be linking to their institutional repository. The first article describes how the data was collected and analyzed and provides basic demographics of the users of library's services. The research questions in this paper were:
In the second paper, the authors look at a subset of the population that could provide the clearest association of library usage and academic outcome: first-year, non-transfer students. This group would have the fewest confounders, such as previous college experience, to muddie the results. They looked at the effects of library usage on both student achievement (grades) and retention. Their statistical analyses was more sophisticated than you typically see in LIS research, using not only t-tests and chi-squared-tests to determine the significance of differences between groups, but also determining the effect size (medium) and multiple linear and logistic regression. They attribute the (relative) richness of this analysis to their own outreach to campus statisticians, and they recommend libraries not try to do it all themselves. Hear, Hear. The value of doing this analysis is that the authors demonstrated the size of the relationships they had found (significantly positive), but also the limitations of such relationships (most correlations were small).
However, with their models, they were able demonstrate significant effect of any library usage on GPA, while controlling for demographic factors. Essentially, users of library service had a GPA 0.23 points higher than non-users, and that usage of the library accounted for 12.4% of the difference between the groups. Given how many factors went into the model (12), that's a bigger chunk than expected (1/12th or 8.3%). The second model broke the services out. Not unexpectedly, these effects were much, much smaller. The only services that showed statistically significant effects were database use, book loans, and workstation logins, and while the size was small, they were services that would be used repeatedly over the course of the semester. The totality of the usage of services represented a larger share of the effect - 13.7%.
Finally, their logistical regression models were conducted to predict student retention based on either usage of any service or usage of specific services. This kind of model demonstrates the strength of the relationships by showing how library services can predict, or explain, outcomes. This is a key aspect of research - can a variable predict a specific outcome? If it can, then it can be used to change the outcome. These models were both significant, even when adjusting for demographic factors. Another important feature of logistic regression analysis is the calculation of the odds ratio (OR). This measures the sized of the effect of the variable on the outcome. In this case, students who used any of the library's services were 1.54 times more likely to continue to the next semester than those who did not. Conversely, few of the individual services showed significant effects on retention; those that did were likely due to small sample sizes (few attendees).
So, what does this all mean? Using these moderately-sophisticated statistical analyses is very much like triangulation - analyzing data from different angles to see the true picture. This picture shows that there appears to be modest relationship between usage of any of the library's services and student achievement and retention. However, picking out which services had the biggest effects is more difficult. The linear model showed database logins, workstation logins and materials circulation as having a small effect; this doesn't show, though, in the logistic model. More evidence, then, is needed.
It is somewhat disappointing that more interpersonal services, such as instruction and consultation, showed much lower effects. This, I imagine, is due in no small part to the size of the data set. Usage of these was services much lower compared to the more self-service, well, services. This could hide any association of the less-used services because of higher standard errors. The problem with studying such low-usage services is selection bias. If this can be controlled, randomly-selecting classes to provide instruction, then the effects should be more significant.
These articles, I think, are invaluable to the efforts of demonstrating value. Like all applied research, it is but a piece in the overall puzzle. It is not sufficient for the argument, but with more such studies filling in the gaps of knowledge, the picture becomes more and more clear. It would be nice, however, if those who are the intended audience of such studies (presumably the campus decision-makers) would show their interest.
- Do sufficient measures exist to determine what services individual library patrons use?
- Do the Libraries reach the majority of students in some way?
- Do students in different colleges use library materials and services in different ways?
With increasing use of methods that capture, essentially, who uses each service, they were able to link with demographic and academic data about each user. The "access points" or services for which data was captured includes:
- How does undergraduate library use compare to that of graduate students?
- material circulation
- ILL requests
- library workstation logins
- usage of electronic resources for those who were off-campus or those who logged into the library workstations
- attendance at workshops
- reference consultations
- course-integrated instruction (through Blackboard)
While not a complete set of data from all library services, this set does represent much of what the library provides. Missing are usage of electronic resources from those on-campus and not using library workstations, in-house usage of materials, brief reference transactions, and visits to the library. The authors admit that such data, especially the last, would be essential for measurement of "library as place", but they express concerns about over-reaching to such an extent that it would affect usage of the very services they would be measuring.
But it's still a big data set - over 1.5 million transactions from over 61,000 unique users. They were able to work with their Office of Institutional Research to get the demographic and academic data. This step has been a common obstacle to such impact research for many libraries, whose own OIR's were reluctant to share the information. The solution seems to be to split the collection of data between the two campus units - library gathers the user identifiers and the OIR provides the demographic/academic data, returning an anonymized data set to the library. This way, neither party should have access to the entire data set, thus securing privacy that much more.
With the complete, anonymized data set, the librarians were able to run correlation analysis to determine if academic achievement was in any way associated with library usage of any kind. Pretty basic...and, to no one's surprise, there was a significant correlation. This is very much in-line with other similar studies, such as the Library Impact Project. I found this part interesting, though (emphasis added):
How frustrating! University administration puts libraries (and others) under pressure to justify our value, our impact on student achievement, then says, "So?"! Maybe this demonstrates a need to find out exactly what measures administrators expect.Already library staff have been able to share this data with University deans and administration and the feedback has been both positive and somewhat unexpected. For example, while University administrators have been enthused by the results, they are also not surprised. It seems intuitive that libraries should be able to demonstrate appropriate levels of usage, and that usage should result in increased academic success.
In the second paper, the authors look at a subset of the population that could provide the clearest association of library usage and academic outcome: first-year, non-transfer students. This group would have the fewest confounders, such as previous college experience, to muddie the results. They looked at the effects of library usage on both student achievement (grades) and retention. Their statistical analyses was more sophisticated than you typically see in LIS research, using not only t-tests and chi-squared-tests to determine the significance of differences between groups, but also determining the effect size (medium) and multiple linear and logistic regression. They attribute the (relative) richness of this analysis to their own outreach to campus statisticians, and they recommend libraries not try to do it all themselves. Hear, Hear. The value of doing this analysis is that the authors demonstrated the size of the relationships they had found (significantly positive), but also the limitations of such relationships (most correlations were small).
However, with their models, they were able demonstrate significant effect of any library usage on GPA, while controlling for demographic factors. Essentially, users of library service had a GPA 0.23 points higher than non-users, and that usage of the library accounted for 12.4% of the difference between the groups. Given how many factors went into the model (12), that's a bigger chunk than expected (1/12th or 8.3%). The second model broke the services out. Not unexpectedly, these effects were much, much smaller. The only services that showed statistically significant effects were database use, book loans, and workstation logins, and while the size was small, they were services that would be used repeatedly over the course of the semester. The totality of the usage of services represented a larger share of the effect - 13.7%.
Finally, their logistical regression models were conducted to predict student retention based on either usage of any service or usage of specific services. This kind of model demonstrates the strength of the relationships by showing how library services can predict, or explain, outcomes. This is a key aspect of research - can a variable predict a specific outcome? If it can, then it can be used to change the outcome. These models were both significant, even when adjusting for demographic factors. Another important feature of logistic regression analysis is the calculation of the odds ratio (OR). This measures the sized of the effect of the variable on the outcome. In this case, students who used any of the library's services were 1.54 times more likely to continue to the next semester than those who did not. Conversely, few of the individual services showed significant effects on retention; those that did were likely due to small sample sizes (few attendees).
So, what does this all mean? Using these moderately-sophisticated statistical analyses is very much like triangulation - analyzing data from different angles to see the true picture. This picture shows that there appears to be modest relationship between usage of any of the library's services and student achievement and retention. However, picking out which services had the biggest effects is more difficult. The linear model showed database logins, workstation logins and materials circulation as having a small effect; this doesn't show, though, in the logistic model. More evidence, then, is needed.
It is somewhat disappointing that more interpersonal services, such as instruction and consultation, showed much lower effects. This, I imagine, is due in no small part to the size of the data set. Usage of these was services much lower compared to the more self-service, well, services. This could hide any association of the less-used services because of higher standard errors. The problem with studying such low-usage services is selection bias. If this can be controlled, randomly-selecting classes to provide instruction, then the effects should be more significant.
These articles, I think, are invaluable to the efforts of demonstrating value. Like all applied research, it is but a piece in the overall puzzle. It is not sufficient for the argument, but with more such studies filling in the gaps of knowledge, the picture becomes more and more clear. It would be nice, however, if those who are the intended audience of such studies (presumably the campus decision-makers) would show their interest.
Subscribe to:
Posts (Atom)