Tuesday, April 1, 2014

Ensuring quality and determining effectiveness: An ELI Focus Session Presentation


As many institutions have invested in faculty development programming and understand that it is pivotal to the success of innovation in course designs and academic programming, there is a need to ensure that the products resulting from these efforts are meeting institutional standards of quality for student learning and other outcomes. We have seen an array of mediated forms of learning (hybrid, blended, flipped, online, self-paced, competency-based, MOOCs, and more) being diffused across campuses and systems, and many of us have been asked to provide evidence of the effectiveness of our faculty development programming to ensure the quality of classes and programs. Administrators are looking for an ROI in faculty development as we are seeing decreases in funding, enrollments, and budgets. This presentation will share an approach to ensuring quality and evaluating the effectiveness of faculty development, including the sharing of resources.

Outcomes: Learn about a life cycle of ensuring quality in faculty development * Identify steps in a backward-design approach to evaluating the effectiveness of faculty development * Share potential resources to use in future efforts 

Link to presentation: http://www.educause.edu/events/online-spring-focus-session-faculty-engagement-and-development/2014/ensuring-quality-and-determining-effectiveness




UWMLTC Faculty Development Evaluations:

Teaching Program:

Support Group:

Teaching Certification for Blended:

Lazirko Award

Peer eval handbook can be found at: http://tinyurl.com/UWMEvalHandbook. A condensed version of the peer evaluation guide can be found at:http://uwmltc.org/?p=3813.

Course evaluation checklist:

Syllabus checklist:

Learning module checklist:

Assessment plan checklist:

ALN 2011, Online Programming Guide:

Tanya Joosten's Blog:

Tanya's Survey Wiki:

POD list of resources (on WikiPODia)

Van Note Chism, N., Holley, M., & Harris, C. (2012, July). Annotated Bibliography on Impact of Educational Development Interventions. Indiana University, Bloomington, IN.

From ELI Learning Circle, Rubrics/Metrics

“Chico model” : __http://www.csuchico.edu/roi/the_rubric.shtml__
Quality Matters : __https://www.qualitymatters.org/__
Guskey’s five levels of prof development __http://www.educationminnesota.org/en/professional-development/tall/5levels.aspx__>>
Also check out the next iteration of the (CSU-Cal State University) Chico Model which is being adopted by the whole 23-campus CSU system. This model is called “QOLT” Quality Online Learning & Teaching: __http://ecatalst.org/our-services/qolt__
Additional resources on ensuring quality in online and blended, including evaluating, please see:__http://eli14.wikispaces.com/Resources__

UCF Pedagogical Practice


ELI Faculty Engagement and Development Resource List



Thursday, March 13, 2014

Learning Technology Roadmap for UW System

What are you doing to plan for your learning technology future?

As a member of the Learn@UW Executive Committee and chair of the Academic Technology Roadmap Task force, we are trying to do some strategic directioning around the use and potential future needs of learning technologies.  I should note it is called the academic technology, blah, blah, but I really prefer the term learning technology when discussing technologies that have the potential when used appropriately to impact student learning.  I am very excited about this process and here is a little more about it:

"With the rapidly changing instructional environment and technology offerings, an academic roadmap is needed to guide the Learn@UW Executive Committee in making key decisions in planning and budget exercise, specifically:

Prioritize financial support for system wide instructional applications
Build capacity and responsiveness for future needs
Develop an understanding of the learning ecosystem within the system"

See http://www.wisconsin.edu/olit/luwexec/projects/ for more details.

We have several things that we are planning to ensure quality in our process and product.  Now, I am not new to this having co-chaired UWM's digital future (teaching and learning track) planning (see digitalfuture.uwm.edu).  Also, I am currently participating in our Chancellor's campus planning, and for a couple years have been participating in State of Wisconsin DPI digital learning planning.

But, I want to know what you all are doing to plan for the future.

So, please share with me by responding to this post or putting on social media (@tjoosten).  I hope to put together a resource to help all of us in education plan for the future.

Tuesday, December 17, 2013

Emerging technological and pedagogical initiatives: A strategic model for pilot, implementation, evaluation, and diffusion

The development of this model began more systematically in 2005 with the student response system project, but since then has been used for an array of projects. This is a working draft and not complete, but since I have been asked to share it more often. I thought I would get it out on the blog.


The design and development of a technology enhanced pedagogical process for implementation on campus

Does the proposed process align with campus goals or is encouraged by campus initiatives?

Is there a national trend highlighting the potential or evidence of effectiveness of the process?

Does the LTC have the resources to implement this process and to yield the desired outcomes (e.g., costs - human/staff time/faculty time, hardware/software, etc.)?


Does the process have the potential to increase the opportunity for active learning or another pedagogically proven method?

Does the process have the potential to increase student learning (through enhancing engagement, interaction, and participation)? impact retention and performance? impact faculty and student satisfaction?

Have a number of faculty and instructors shown interest in the proposed process?

Are there 2-3 LTC staff members who have interest in exploring and advocating for this proposed process?

Had the LTC identified an effective pedagogical model? or will an exploratory analysis yield a pedagogical model which is generalizable or scalable and yield desired outcomes (ie student learning)?


Who will participate, support, or provide/yield resources to the success of the implementation? How?

Departmental instructional support
Student services
Dept/college admin
Outside organizations

Pedagogical Models

How, specifically, can the proposed technology facilitate the desired outcomes?

Models and cases should be developed based on our own LTC staff members' implementation in his/her courses, from an outside faculty and case, or from a leading/innovative use on campus.  These cases should highlight the assessment of student learning, providing student feedback, and student support leading to high satisfaction.

Is this model going to result in the greatest pedagogical impact? Why?

What research/literature supports this use of the technology to facilitate the proposed process?

Technology Selection

Which technology or technologies is most appropriate to meet our needs?

Technology selection process can include campus product demonstration, stakeholder feedback solicitation, technology evaluation.

Potential evaluation criteria -
Ease of use - Can most instructors and all LTC staff use this product?
Easy for faculty to learn
Easy for students to learn
Training and support - Does the technology/vendor allow for self-support or can staff be easily trained/supported?
Satisfaction of current users - Are current technology users satisfied with the technology?
Validity - Does the technology do what it is supposed to do?
Reliability - Does it work on a consistent basis?
Interaction/Active Learning- Does the technology increase student/instructor participation, communication, and engagement?
Acquisition - Is it affordable, easy to purchase, and set-up?
Vendor - Is the company well established with a good reputation?
Hardware/software readily accessible
Cost for student
Number of users
508 compliant
Student activity fully documented during use
Good pedagogical examples ready to hand
Meets learning objectives and activities across many disciplines
Supports collaboration
Supports critical thinking
Supports student engagement
Supports student contribution to course content

Faculty Selection

Faculty recruitment selection can be conducted using a competitive process and Request for Proposals.
Potential evaluation criteria -
Do the faculty members represent an array of disciplines? course level? course size?
Do they have a sound pedagogical practice? assessment practice?
Is the faculty member a current technology user?
Is the faculty member a potential change agent?


Mixed methods
Qualtrics survey development
Sample instruments are available at: http://mysurveys.wikispaces.com
Likert and open-ended
Faculty and student surveys
Faculty focus groups and debriefing sessions
Faculty narratives
Factor analysis
Survey reliability
Thematic analysis
Descriptive results for findings summary and slides
Multivariate analysis for publication

STAGE 4: SHARING/Dissemination

Sample information
Findings summary for general community (blog)
Slides for meetings, including 1 bar chart per variable with 3 items for meetings and conferences
Determine potential publications
Determine conference venues
Determine UWM, UW System meeting/conference venues

EDUCAUSE, ELI, Annual, Midwest Regional
Sloan-C ET4Online, Blended, ALN
DTL, Madison
D2L Fusion

Books, Book chapters
Journals, JALN, etc.


Improve faculty development training and workshops
Include discussion of findings at the beginning of workshops to
1.) Get buy-in on using technology and pedagogy
2.) Build trust by recommending evidence based practices
3.) Develop identity for LTC in research

Need time and resources
1.) Should we continue the project?
2.) If so, how can we scale support - workshops, web-based, etc.
3.) At what interval will we evaluate

Need additional strategic vision on temporal plan for re-evaluation of projects

Tuesday, October 29, 2013

Case Study: A scenario-based form of assessment

I have been lucky to teach in organization communication, which has a wealth of case studies that I can use in my teaching as well as a form of assessment.  Case studies are a key pedagogical tool and form of assessment.  Although I have developed some brief case studies, which may border more on the form of a high level taxonomy (problem-solving or interpretive) multiple choice item, I have seldom had to develop my own case studies.  Also, I have appreciated in faculty development in using technology in the classroom, case study as a form of professional development (see the book, Team-based learning, Michaelson, Fink).

Recently, in working with a group of teachers, I realized that not everyone has access to the wealth of case studies of which I do.  Also, although I have developed some, I didn't really have a guide, template, or framework.  Therefore, this post is to start to do actually that...gather resources in order to develop a practical guide for faculty and teachers to use to develop case studies for their courses, which can be used for teaching or for assessment.

Potential resources:


Please share your resources here, or please share them on Twitter using the hashtag #casestudy and tag me @tjoosten.


Wednesday, October 23, 2013

Pearson: Social Media for Teaching and Learning #smteachlearn

I was very honored to be invited to present as the closing keynote at a Pearson event on social media. The audience was a group of educators that were actively using social media for teaching and learning, which made this most existing for me.  I love having the opportunity to share with folks that are actually using social media already.  It usually means I am going to learn something new today!

Newark Airport, Gate to Boston
The downside to this is that I was going to have to leave #edu13, EDCUAUSE Annual conference, and take a red eye to Boston (via Newark).  This is what the Newark airport at my gate to Boston looks like at 4:30am (which was 1:30am in Anaheim) as I wait for the 6am to Boston. The great news is the Pearson really took care of me so when I got to the hotel in Boston, I was able to catch a few Zzzzzsss.

Conference book
After a few hours of sleep, I started reviewing my presentation for the Pearson Social Media for Teaching and Learning event.  However, I was very challenged to give the presentation that I had intended to give coming off of 4 days of great discussions around social media, ensuring quality in online teaching, piloting new technologies, and using OER.  So, of course, I started playing around with my presentation a bit in the 11th hour.  At the end of the day, I might have done the unthinkable and openly talked about the move to OER at a publisher conference, but I know for a fact that Pearson and others are less in the publishing game these days and more in the learning game.  I think this is a good thing.

And, yes, I might have made another flaw by playing a YouTube video that at the end said STFU, but it was hilarious and timely ad the end of a long day.

I applaud Pearson for putting on a conference on social media, which is almost a competitor in some ways and has changed their business first hand.  At the same time, I am always questioning the motives of vendor to bring educators together to discuss technology's impact on teaching and learning.

Either way, I got to meet some really great new folks and see some old friends I hadn't seen in a while.  Anything that brings a bunch of educators together to talk about improving student learning through the use of technology can't be all bad.   :D

EDUCAUSE Annual Conference 2013 in Anaheim #edu13

As a follow-up to "Come see me at EDUCAUSE," I wanted to share out some resources from my presentations as well as some resources that I gathered at the conference.

Storify of tweets for #edu13 and #edusocmedia tell a bit of the story ;)

Social Media for Teaching and Learning (separate registration required) with my colleague Laura Pasquini, @laurapasquini
EDUCAUSE Virtual Annual Conference 2013, October 2013
Tuesday Morning

Why would you implement any learning technology in your classroom? Because it's cool? Errr...wrong. Come find out about what your pedagogical needs in the classroom may be and how you can use different social media with different characteristics to meet those needs. We will develop specific strategies and materials to get you well on your way to becoming the innovative teacher you always knew you could be.

Resource: http://tinyurl.com/edu13edusocmedia

Part 1: http://educause.acms.com/p6l52fazz1b/
Part 2: http://educause.acms.com/p89r6qnxlc2/

Ensuring Quality in Online and Blended Programs (separate registration required) with my colleague Dylan Barth, @dylanbarth.
EDUCAUSE Face-to-Face Annual Conference 2013, October 2013
Tuesday Afternoon

As the demand for blended and online learning opportunities increases, so does the need to ensure the quality. A recent study reported that "the number of students taking online courses will grow by 22.8% and that those taking blended courses will grow even more over the next 2 years" (Picciano et al. 2012, 128). With the increase in the diffusion of blended and online programming across higher educational institutions, stakeholders are looking for ways to ensure the quality of the student experience. This workshop will provide a collaborative and interactive opportunity to connect with colleagues to consider and construct how the quality of blended and online programs can be ensured through faculty development and training, evidence of instructional competence, recognition of excellence, constructive evaluation and feedback, and community-building opportunities among instructors and staff.

Resource: http://edu13.wikispaces.com

Prepare for Lift-Off: Becoming a Successful IT Pilot Site, panel with my colleagues -- Amy Collier, @amcollier, and George Veletsianos, @veletsianos
EDUCAUSE Virtual Annual Conference 2013, October 2013
Wednesday morning, Learning Theater
Wednesday afternoon, virtual

Your campus is an innovator in many ways, and you've been approached to be a pilot site for a new campus IT product. You'd like to say yes to the idea, but you're not sure you have the infrastructure to make it work. Join a panel of your university colleagues to learn the ropes and discover what it takes to successfully deliver and host technology pilots on your campus. The panelists will offer a dynamic conversation on the importance of stakeholder involvement, faculty engagement and selection, faculty development and support, technical infrastructure, student support, research and evaluation, and critical steps your institution needs to take to ensure your pilot not only flies but soars.

Social Media Constituent Group with my co-lead, @micala (Shannon Ritter)
EDUCAUSE Face-to-Face Annual Conference 2013, October 2013
Thursday morning

We will discuss using social media to increase communication, build online learning community, create connections with the instructor and among students, and enhance engagement. We again will focus on harnessing the capacity of social media and look to members to bring their experiences to share. We will explore the implications of the integration of social media throughout higher education for an array of purposes (e.g., teaching, recruiting, advising, and more). Social media for our purposes is defined as media that's used to build social networks and connections for sharing information through a mediated channel.

Resource: https://docs.google.com/document/d/18JInJuZHOkoIUjalRioC5ATyULX6VLiacb-2E_Y3x2w/edit


Exploring Ginkgotree: Increasing Access, Engagement, and Learning with OER with my colleague, Dylan Barth, @dylanbarth, @ginkgotree
EDUCAUSE Face-to-Face Annual Conference 2013, October 2013
Thursday afternoon

University of Wisconsin–Milwaukee will discuss its partnership with Ginkgotree to explore the efficacy of their application to curate digital content for creating textbook alternatives that improve student outcomes.

Consider faculty development and training in using Ginkgotree and OER | Learn about institutional processes to launch a Ginkgotree pilot | Explore challenges and opportunities experienced by faculty and students

Resource:  To be posted by November 1.  Report on thematic analysis of faculty focus groups and survey as well as lessons learned from operations side of pilot.

For more info on the sessions, check out:

Monday, October 21, 2013

A star

A link between atmospheric sciences and social sciences...yup, I am a nerd.  In deciding how to excel in life,

"The star's internal pressure prevents it from collapsing further under its own gravity."  Wikipedia, Star

Tuesday, October 8, 2013

Educational technology cannot be created in a vacuum

I was very excited this morning to get the news from George @Veletsianos that our #sxswedu panel had been voted on and won!  Meaning, we will be presenting at SXSWedu in Austin, TX sometime between March 3rd and March 6th.

Our session:

"Startups should talk with researchers & educators"

Flickr souvenirsofcanada
Education is facing numerous challenges. Educational technology startups promise solutions. However, entrepreneurs seem to disregard the knowledge that educators and researchers have amassed that can help startups address these challenges, or, at least, help them avoid repeating the mistakes of the past. At the same time, we were astounded by the lack of educators and researchers that were sharing their knowledge at last year’s SXSWedu conference. The event felt more like a vendor gathering than what the SXSWedu website describes as "meaningful conversation and collaboration around promising practices and tools for improved learning.” If we want meaningful and transformational change in how we do education, it is imperative for entrepreneurs and educators/researchers to converse. In this interactive panel, we will discuss how educators/researchers can help startups improve their products and answer questions pertaining to education research, how people learn, and classroom practice. - See more at:

See: http://panelpicker.sxsw.com/vote/21723#sthash.54YgfsPB.dpuf or http://panelpicker.sxsw.com/vote/21723


Flickr souvenirsofcanada
Last year, I was very lucky that my friend and colleague, @laurapasquini, invited me to sit on a panel about social media.

It was my first time at any #sxsw event, and I was very excited to attend a new conference hoping for new discussions and conversations that would help drive education technology development in higher education.  I was a bit overwhelmed at the number of tech companies, start-ups, in attendance.  As the educators/researchers starting instantly connecting with each other, I was amazed at how little of a group of us there was at the conference. For something with "edu" in the title, I wasn't seeing a lot of "edu" at the conference or involved in the dialogue.

Recently, I have become more focused on partnering with vendors to help shape the technologies that meet our needs in edu and better identify the ways in with educator/researchers can collaborate and partner with vendors, ed tech start-ups to develop applications.  Also, we have a vision and strategy and are looking for specific technological solutions to help us fulfill goals.  Many of these solutions are in their infant stages.  With the speed of change these days, we no longer have the luxury of waiting for applications to hit the market, be vetted by an array of educational institutions, and a finalized product to be disseminated throughout higher ed.  We are at a point, that educational institutions, educators, and researchers need to be collaborating with tech companies earlier in the product development cycle.

How many actually understand the importance of these early collaborations?  First, folks from all sides need to understand that not only can we help each other, we need other.  We need to understand what we can each bring to the table and how to work effectively with each other.

Education institutions have been working with vendors for years.  We understand how instructors need to be supported and how students need to be supported.  We have experience with several models of vendor support for instructors and students as well as university models for instructor and student support.  Also, many of these processes are pretty standardized at this point.  We know how to align technologies with our strategic plans, justify our needs for the technology, engage student and faculty, support student and instructors, implement and diffuse the solutions, and evaluate the effectiveness of solutions.

Educational institutions bring a wealth of knowledge of providing research of the effectiveness of a technology solution on teaching and learning.  We have theoretical frameworks and methodological practices, including instrument development, data gathering, and data analyses expertise and resources.  We have avenues to disseminate these findings.

These findings and experiences can help edu tech start-ups improve their support processes, identify problems, highlight product features needed by educators, and prioritize problem reports and feature requests.  Most importantly, researchers can provide evidence for whether or not they actually improve student outcomes.  It is ALL about the student learning, is it not?

At the end of the day, I don't want to hear a salesperson or account manager form technology company X telling my about a product, I want I a colleague from a fellow institution to share with me their experience using the technology.  I want to know what need the application is meeting, what support needs they experienced, and whether it works or not.

So, my SXSWedu 2013 experience not only resulted in this panel, it resulted in my continuing my work in this area of need -- how to connect these companies with educators/researchers, so as a shameless plug, but yet important to my plight, the following #EDU13 sessions, EDUCAUSE annual conference presentations, hit on the same topics.

Make sure to check out my post, Come See Me at EDUCAUSE!

Flickr souvenirsofcanada
Specifically, here are a couple presentations that hit on the topic:

Prepare for Lift-Off: Becoming a Successful IT Pilot Site

Wednesday, October 16, 2013 9:40am-10:20am 
Wednesday, October 16, 2013 3:40pm-4:30pm

Tanya Joosten @tjoosten, George Veletsianos @veletsianos, Amy Collier @amcollier

Exploring Ginkgotree: Increasing Access, Engagement, and Learning

Thursday, October 17, 2013 2:40pm-3:30pm

Tanya Joosten @tjoosten, Dylan Barth @dylanbarth, Ginkgotree @ginkgotree

So, what are your thoughts?  What is the relationship between edu vendors and tech starts ups and edu (institutions, researchers, and educators)?

It is not all about CIOs and who can authorize the purchase ;)

Monday, September 23, 2013

Social Media Constituent Group (SMCG) at #EDU13

Good afternoon, Social Media Constituent Group (SMCG) members!

You are receiving this because you are subscribed to the EDUCAUSE SMCG email list.  It is that time again and we are planning for the SMCG meeting in Anaheim at the EDUCAUSE Annual Conference, #edu13.  We have a couple requests to help us along:

1.) Complete this survey (or the first question, at least) so we can learn more about where we are in higher education in our uses of social media and what are the "pressing issues: https://new.qualtrics.com/SE/?SID=SV_aeH5HHWSYMCzxCl.

2.) Hangout with us as we discuss -- how we should our time during our SMCG slot -- on October 10th, 12PM CST/1PM EST.  We will email a link to our "Hangout On Air" 15 minutes before the meeting starts, or connect with us on G+, and we will invite you.  Mark your calendars!

3.) Tweet your ideas for the SMCG meeting, share recent publications or resources, or #edu13 social media conference sessions using our hashtag #edusocmedia.

4.) Join us in Anaheim, Thursday, October 17th, 8:00 AM - 8:50 AM PST, Meeting Room 210D.  Live tweeters needed! ;)

Tanya and Shannon

Shannon and Tanya at EDUCAUSE

Don't forget to connect with us:

Tanya Joosten @tjoosten, juice.gyoza@gmail.com

Shannon Ritter @micala, micala@gmail.com

Come see me at EDUCAUSE!

Guess who is excited!?

EDUCAUSE Annual Conference in Anaheim, CA, kicks off on October 15th.  I am looking forward to meeting new people, gathering lots of information, and hanging out with old friends.  Also, I get the opportunity to share some of what I have been working on the past year.

Image of EDUCAUSE Presentations

I have two workshop on Tuesday.  One virtual workshop in the morning on social media with my colleague Laura Pasquini, @laurapasquini, and one f2f workshop in the afternoon on ensuring quality in online and blended programs with my colleague Dylan Barth, @dylanbarth.

Wednesday, it is the day of discussing vendor relations.  On a panel with my colleagues -- Amy Collier, @amcollier, and George Veletsianos, @veletsianos -- I will be discussing considerations when technology companies and educators partner to pilot innovations.  This will be a f2f session in the morning and a virtual session in the afternoon.

On Thursday morning, bright and early (8am - yikes), we will be talking social media at the constituent group meeting (bloody mary's are optional) with my lovely co-lead, @micala (Shannon Ritter).  Finally, I wrap up the day talking with my friends at Ginkgotree, @ginkgotree, about our pilot of their product this fall.

Please come check out one of my sessions and say, "hi!"

For more info on the sessions, check out:


1.) Tweet about social media using #edusocmedia
2.) Tweet about the conference using #edu13
3.) Tuesday social event is TechSmith.  More to be announced at a future date ;)
4.) My phone will likely die each day, so ask @robin2go for directions to find me.