Seeing Data

Part of what makes what I do so interesting is the interconnectedness of the moving parts of the organization. My group, Teaching and Learning with Technology (TLT), has many moving parts and a handful of groups within it that all do different, yet similar things. Our Classroom and Lab Computing (CLC) team focuses time on our physical spaces and the infrastructure that supports them. Education Technology Services (ETS) is a different type of organization in that they not only evaluate, test, and recommend technology solutions for faculty, staff, and students but they also look to unpack the affordances of those technologies. ITS Training Services spends most of their time designing and delivering training to faculty, staff, and students but also manage big projects that deliver all sorts of other services. WebLion is a team that focuses intensely on the overall processes inherent in the design, development, and deployment of large organizational websites. What amazes me is that each of these groups are all part of a larger value chain of sorts that serves our University very well.

They also all produce lots and lots of data. Some of that data comes from the services that we offer, while a whole bunch of it comes from the questions we ask of our audiences. A couple of examples of service level data might be stuff that our Adobe Connect service collects for us, or the data we get each time a student prints in one of our labs, or each time an application is launched. With this kind of data we don’t know what happens in an Adobe Connect meeting, or know what the application does once launched, or the contents of the pages printed (we’d need to do a different kind of analysis to get at that) but we do get clues that help us ask better questions that presumably help us make better decisions. From where I sit the days of, “I betcha …” planning are long gone.

To this end we’ve embarked on trying to make sense of the data we have in new ways — visual ways. A small team in TLT has been working at using the Roambi platform to do just that, visualize otherwise flat data tables to help us make better sense of what is being gathered. The unfortunate thing about this blog post is that you can’t get a sense of what it is like to not only see your data, but to be able to touch it and manipulate it in real time. The screenshot below is a simple representation of the percentage of overall pages printed during the 2011 academic year (Spring, Summer, and Fall) from the College of the Liberal Arts faculty, staff, and students using our managed lab environments.

Printing Data

Now taken by itself this is an interesting piece of information that lets you see that we print quite a bit still in total and that the College of the Liberal Arts prints about 15% of the total number of pages here. But if you take that data and mash it up with our ability to look at it from the departmental level you can begin to make sense of how to address it as a problem to be solved. Printing is expensive and while we do all sorts of really great things to be eco-friendly, it isn’t the best thing for the environment. When you look at the drill down at the departmental level you can pinpoint specific programs that print much more than others. When you do that you can work at a level where some sort of intervention can be applied.

This is where the organizational integration starts to really become powerful. Having the printing data and the departmental data mashed together I can sit down with the unit level directors and begin to construct a strategy to change the overall behavior in a positive way. I can now work with instructional designers in other parts of TLT to construct a workshop for faculty on digital assessment strategies, create learning opportunities for students to understand how technology can be used in the writing process to eliminate passing drafts around, and look at new software that enables new workflows between those audiences. Then we can easily measure the pre and post states to see if our intervention might be working.

Another example that happened not too long ago … we had a meeting to discuss changes with one of our smaller public labs on campus. Essentially we were asked what the impact of moving this small lab might have on students. This happens all the time — construction needs to happen for all sorts of reasons and typically they are good reasons. Well the meeting started and I was able to quickly show just how intensely popular this out of the way location really is. Needless to say the person we were meeting with was blown away and offered new space to better meet the needs of students. It is hard to pack this much information into a readable spreadsheet. The visualization below shows all sorts data represented in an easy to read format … I snapped a single day that represents 3,917 unique userids that logged into the machines in this small out of the way location. By having this kind of data available and readable we can instantly see trends in use and share that in a meaningful way.

Lab Data

We are just at the beginning stages of this approach and haven’t figured all of the details out. But as we go forward we know that using our data in this way could be truly part of a transformative way to get at the future states of our services, our spaces, and the ways we plan for them.

Organizational Frameworks

I have been in my role as senior director of Teaching and Learning with Technology at Penn State since November 15, 2010 and in those nine months I have been working to better understand the organization both in terms of its external requirements and the overall internal dynamics. I feel very lucky across several dimensions in that I have a great leadership team in place that has rolled up its sleeves with me to help explain the various functions inside their own units and who have also embraced this idea that we have a real opportunity to rethink how we work together.  Another critical factor at play here is that I still have access to the person who built this organization and was a huge driving factor in the creation of such a robust teaching and learning with technology ecosystem here at the University.  What that affords me is an opportunity to grow into my role and have people to lean on in all directions — it has been critical as I work each day to better understand the overall depth and breadth of TLT and its overall role here at the University.

As part of this process I challenged my leadership team to come together and help me rethink the way we work together and present ourselves to both the on and above campus audiences we serve. I’ve pressed them into the idea that we can no longer do what we need to while being a handful of individual organizations — we need to think, talk, and act as one TLT. This idea, that we are better together than as separate and vertical organizations is something I believe very strongly in. My push is that we need to see ourselves as a horizontally integrated organization — an organization where our teams leverage the talents across the lines of the individual groups. I say this because I truly believe TLT has been constructed in a very intelligent and thoughtful way .. we are an organization that has each piece of the puzzle as it relates to envisioning, implementing, and supporting large and small scale technologies that influence teaching and learning.  What I mean is that we have a value chain of sorts in place that allows us to actively investigate new and emerging technologies and practices with an incredible amount of agility in Education Technology Services, we have the ability to install, manage, shape, and support all that activity in both physical and virtual ways through the Classroom and Lab Computing team, have the ability to drive adoption and appropriate use of technology through Training Services, and can work to communicate much of it on the web through standards-based accessible web presences powered by WebLion.  These organizations need to compliment one another as we work to deliver the kinds of services our audiences need and want.  They need to act as One TLT.

Tlt view

This perspective, when implemented, allows for our project teams to organize around successful implementations in ways we may not have considered in the past.  As a recent example, when we set out to replace our student response system, we didn’t just turn to one of the organizations to make a technology decision, we assembled a team that included not only purely technical people who focused on the integration issues, but also an instructional designer to investigate and document teaching practice, a trainer to construct training opportunities from the start, and communication people to share progress openly as we drove towards selection and implementation. Sounds simple — and it is conceptually, but the act of actually making that the new framework in how we do work is the complicated thing.  We can’t live in a world where any one of the organizations within TLT does its own thing from end to end — end to end requires the skills only available when you look across TLT from a horizontal perspective.

This is also true in the way we need to begin to represent ourselves as well. One of the things we have done every year I have been a part of this organization is write an annual report. Typically the responsibility to construct the report would fall directly on the shoulders of the director in each of the primary groups. What this meant was that the report read more like four or five different reports under one cover page. This lead to some strange reporting — CLC and ETS would both report on projects they were involved in (like the Media Commons) and often times the data shared might be slightly contradictory and tell two different stories. What we set out to do this year was much different — we wanted the report to represent our thinking as it related to TLT. It honestly took quite a bit longer than I expected to work through the thinking, but in the end I am left very proud of what we developed and I believe it will be the blueprint that much of our work will follow over the next couple of years. Last year’s report was nearly 140 pages, this year’s report is 23 in total. (What follows is mostly for me, so I can capture the process of creating it while it is still relatively fresh in my head.)

Several months ago I started the conversation about the annual report with the TLT leadership team and we all agreed we wanted something that could more effectively speak to who we were as a collective.  Our first step was to take the 140 page report and break each headline into a blog post. Each post included the title of the section and a short description of the initiative.  The blog gave us a multipage digital representation of a static document.  We fully intended to use that as a platform to allow all of TLT to vote on the most important initiatives to form the basis of the report.

Annual report blog sm

Bu once the blog was in place and we looked at it, something different ended up happening. I walked into my colleague, Derek Gittler’s office and he had taken every headline and placed them on sticky notes. He even color coded them based on what I’ll call the organizational owner.  We looked at it and were at once shocked at the overlap and the emergence of themes. I was able to easily construct a handful of themes that highlighted what our largest and most impactful initiatives are. Within the hour we had taken the blog built around what should be a hidden org structure from our report and turned it into a thematic representation of TLT.

White board sm

Once the themes emerged, I was able to assemble a Keynote presentation for the leadership team so we could drive towards consensus as a team. The presentation outlined the themes and how our projects and initiatives come together to tell an amazing story of the organization. A story that allowed us to share short details about how TLT focuses intense energy around:

  • Teaching, Learning, and Collaborative Spaces
  • Collaborative Platforms for Teaching and Learning
  • ANGEL and the Future of the Course Management System
  • Enriching the Community
  • Engaging the Community
  • TLT Events
  • TLT Research and Assessment
  • The Future of the Web
  • Conservation in TLT

The themes turned into a series of wiki pages that the leadership team constructed from the outline from the whiteboard. From there the leadership team took a couple of days to gather the appropriate data from each item and write it up in the wiki. I was able to leverage the wiki and write the final report, with narrative in less than 24 hours. Once the communication team did the editing the report came together remarkably fast — after the months of preparation and discourse.

I know it seems almost silly, but for the first time I can look at TLT and see how we work together to provide services and opportunities that truly supports our mission to guide the University in the appropriate use of technology to enrich teaching and learning. When you read through the TLT Annual Report for 2010 I hope you can see that what we are attempting to do is provide not only a new way to communicate our accomplishments, but a new willingness to address our own organizational framework to better serve those who depend on us the most. Maybe taking a few months to craft an annual report seems extreme, but in this case I honestly feel the work that we did here will provide the foundation for how we work together going forward. It is something I am very proud of.

This post also appears at my PSU Blog. Sorry for any multiple linking.