Thursday, October 16, 2014

Student perceptions of course design

This blog post is a summary of my efforts to develop a survey to be delivered to students to evaluate course design.  The instrument is developed, in part, from the UC Chico Rubric, Quality Matters, and our own rubrics and checklists used at UWM.  If you have an instrument to share or additions, please comment to this post.

I will continue to update as I refine the instrument and will report the stats on the instrument once I administer it this fall.  Although originally intended for online and maybe even blended courses, this will be delivered to f2f courses as well to compare course design across mediums.  I am also developing an instrument for instructors and staff as well.

If you are interested in delivery my instrument and collaborating on research, please let me know.  Would be great to get some cross-institutional data.  In this investigation, I am also examining interactivity (engagement, community, presence, etc.), learning, performance, and satisfaction.


I had adequate support in completing my activities.
I received support materials prior to starting the class activities.
I had information for whom to contact if I needed support.
The syllabus was easily located and included course objectives and completion requirements.
Expectations of students’ participation were included in the syllabus or in D2L.
A clear timeline or schedule for face-to-face and online activities was shared.
I received information on the availability of and turnaround time for contact with instructor.
The introductory explanations on the class were clear.

The course was well-organized.
Course content is “chunked” for more manageable learning
Course content is organized in a logical format
Topics are clearly identified and subtopics are related to topics
I understood all components of the activities.
The instructions for the class were clear.
Course schedule is available in a printer-friendly format for student convenience
I understood the layout of course.
Language of written material is friendly and supportive.
The goals of the course were clearly defined.
The goals of the activities were clearly defined.
The method of grading my performance was clear.
I understood what was expected of me.
Sentences and paragraphs were brief and easy to understand.

Instructional design and delivery
I had the opportunity to introduce myself to others.
I completed an “Ice-breaker” activity or other orientation session to get acquainted
I was prompted by my instructor to expand on relevant points
Each reading assignment and each activity matches a learning objective
Activities have an assessment piece that links to a learning objective
Tasks and activities are designated as synchronous or asynchronous
Summary provided frequently, particularly at the end of topics, to reinforce learning expectations for that module

Assessment and Evaluation
The instructor shared the criteria used to assess participation discussions
I was not assessed solely on tests/quizzes
I was provided ample opportunity to demonstrate proficiency in different ways
I received rich and rapid feedback
I received frequent and substantial feedback from the instructor
The instructor provided samples of assignments illustrate instructor’s expectations
I received detailed instructions and tips for completing assignments
The instructor provided due dates for all assignments
Rubrics for all assignments identify assessment guidelines were provided
A grading scale was shared by the instructor
Peer review opportunities were available
I had an opportunity to apply rubric to my own work
My input on the class was sought by the instructor

I will look to integrate Kelvin's, Penny's, and Barry's work.

Other resources include:

UCF Pedagogical practice | | Contact @kthompso

My creations with my colleagues through the years...

UWM Preso, Ensuring Quality

UWM's Course Evaluation Checklist

UWM Instructor's Guide to Evaluation A condensed version of the peer evaluation guide can be found at:

UWM Syllabus checklist:

UWM Learning module checklist:

UWM Assessment plan checklist:

Sunday, September 28, 2014

Heading to the EDUCAUSE Annual Conference

Well, what will I be doing at EDUCAUSE...this is mainly the "public" places where you can find me.  Don't forget to follow me @tjoosten,, or follow the educators social media hashtag on Twitter at #edusocmedia.

FIRST, hangout with Shannon Ritter, @micala, and I has we facilitate a FUNCONFERENCE 8am Monday morning.  Believe me, we are worth it, even at 8am!

Monday morning seminar (f2f): Social Media Constituent Group Unconference (separate registration is required)


The Social Media Constituent Group invites anyone interested in the use of social media in higher education to join the conversation and discussion during our seminar without any constructed agenda or specific outline. Come join our community of colleagues interested in best practices for the use of social media in higher education.

OUTCOMES: Develop connections with colleagues in this constituent group and become active community members * Gain experience and knowledge related to the use of social media in higher education * Build skills and connections to continue being active community members during and after the annual conference

Monday, September 29, 2014 8:00am-11:30am | Room Meeting Room W310A/B

Register at: or onsite in Orlando

SECOND, wether you are in Orlando or not, sign up to join Laura Pasquini, @laurapasqui, and I as we discuss guidelines, policies, and practices of social media in Adobe Connect.

Monday afternoon seminar (virtual): The State of Social Media Guidance: Implications of Guidelines, Policies, and Practice in Higher Education (separate registration required)

Higher education institutions are using social media to communicate and engage their campus community; however, very few are examining the impact and implications of social media guidance. From research to practical applications, this seminar will detail the current state of social media guidance in postsecondary education and identify key elements of guiding principles that offer suggestions for student support, teaching, training and development, research, infrastructure, and more.

Participants will be able to: Articulate current higher education social media guidelines and policy document trends, categories, themes, and patterns emerging from research * Identify practical components for effective social media guidelines for campuses * Create meaningful guidelines and policies to positively impact teaching, learning, research, and development at your institution

Monday, Sep 29th, 2014, 12:30 PM - 4:00 PM Eastern Time

Register at:

Twitter hashtags: #edusocmedia and #edu14

THIRD, come chat about social media in higher ed at the smcg meeting.  Get all your questions answered or find out that lots of us have the same ones ;) 

Tuesday afternoon constiuent group meeting: Social Media Meetup!
Free - no additional registration needed

Social Media Constituent Group meeting.

The Social Media Constituent Group invites anyone interested in the use of social media to be a part of our "unconference" session to discuss, brainstorm, collaborate, and learn about how your colleagues across higher education are using these tools and technologies. The agenda for the unconference will be created by participants at the start of the session and discussion will be facilitated by the co-leaders of the EDUCAUSE Social Media Constituent Group. Attendees will take responsibility for their own learning and guide the discussion around topics and areas of interest. We are an active, engaged group of higher education professionals and invite you to be a part of our unconference, join our group, and share your best practices.

Tuesday, Sep 30th, 2014, 3:40 PM - 4:30 PM, Meeting Room W308C/D

FOURTH, most importantly, join us for a social media #tweetup. It is a great way to meet colleagues who are interested in using social media or just to meet other folks in Higher Ed that use technology.  Devices and selfies are welcome, maybe even encourage.  Drinks are required ;)    
Tuesday social:

After the CG meeting = #edusocmedia Happy Hour Tweetup  – Tuesday (9/30) @ 5:30-6:30p David’s Club Bar & Grill @ Hilton (right next to conference center).

David's is a bar in the Hilton.  We just chose it since most of us are at the Hilton.  We will had there after dropping bags in our room after the CG meeting.  Once we leave here, we will head to a vendor party that is a nice time every year. 

FIFTH, Tuesday night vendor social event:

Tech smith party open to all - sign up with their eventbrite.  Who doesn't use snag it, jing, or camtasia, but we are always thanking to hang out with the great hosts of Tech Smith.  

FINALLY, Late night shenanigans, yet to be determined, but tends to having dancing involved and maybe that dancing is on a stage or in a street with Bumble Bee, the transformer, or Batman. Nope, I am not being sarcastic this time ;) 

See you there!

...Oh, and if you haven't read it -- read it!

Sunday, September 14, 2014

Top Online Library Resources

A resource for student research projects

CC gcoupe Flickr

Google Scholar:
Internet Public Library:
Library Spot:
Univ. of CA:
Harvard College Library:
Carnegie Melon:
Penn State:

Tuesday, June 10, 2014

UW System Roadmap Summit

I have spent about 9 months along with my several UW System Learn@UW Executive Committee colleagues having extensive discussions and scholarly debate about developing strategic directioning and vision for UW System academic and learning technologies.  Our discussions led to the planning and launching of a UW System Roadmap Summit, which at a minimum feels like a success since many attendees felt as if it was needed and it should be happening on a more consistent basis (considering the last one was about 15-20 years ago according to reports).
Roadmap Summit Pre-Event

As the chair of this Roadmap process and one of the folks who felt strongly for developing a "plan" or at a minimum better understanding System needs to help us make better decisions about the acquisition of learning technologies, I invested a significant amount of time in this process.  Rather than this plan be
 "expert" drive or developed by a small group of people in a somewhat closed system/room, I wanted to make sure that we had rigorous methods and engaged stakeholders throughout the System.  With that, I have a few reflections.

What was one thing you really liked?

I really loved seeing representatives from each and every campus in one room, but a small enough group where most still felt they could have a voice (we never wanted more than 75 people and physical limitations of rooms can be a good thing).   Not only did we see diversity in the campuses represented, we saw diversity in the constituents.  The attendees were Provost nominated for the most part, so we saw folks from all different divisions of campus.  There was definitely an academic and faculty voice along with an IT and service or support voice.  Interesting enough, sometimes resulted in some decent debate and opportunities for perspective taking.  I am big on bringing together heterogeneous groups since the research tells us, they will lead to an optimal product in the long run (if 100 other contingencies are accounted for).

What was on thing you would have improved?

I worked very hard to make sure the agenda was active and interactive.  I didn't want to focus on anyone person sharing more than 15 minutes.  I wanted to value folks time to come together f2f, provide some time for sharing but not let it dominated, and focus our attentions on discussions and interactivity.   We realized that folks wanted to share out, but this can be an abyss -- everyone wanting to share everything and be heard resulting in a lot of bored people in the room.  Therefore, we pulled themes from survey data, interviews, and reports, and organized lightning round panels around these themes.  I wish we would have planned more time for these panels to facilitate discussion about leveraging System resources to improve teaching and learning, scale support and services, and reduce costs.  Moving forward, I would like to work to facilitate these discussions.  For instance, about half of the campuses in UW System are working on Active Learning Classrooms, negotiating contracts separately, planning one-off evaluations, if at all, etc.  I would be nice if we were working together to share in this knowledge and working together to maneuver vendors.

What was one thing that really surprised you?

Many of the folks mistook the goal of the summit to announce some decision regarding a move away from our largest learning technology enterprise systems, the Learning Management System, D2L.  Many wanted to know when we were leaving or why we were leaving.  Others felt that the Summit was a smoke shield or opportunity for us to get buy in to leave D2L.  It seemed like many didn't truly believe we were just there to understand where we were and where we wanted to go -- as a System.  There was and has been no decision made nor will their be without numerous others steps being taken.

UW LaX Infrastructure Panel
Although I was surprised that folks thought the efforts might be disingenuous, I also have seen in higher education that decisions are made with a small group of people in a broom closet, and after the fact, stakeholders are engaged in efforts of buy-in rather than truly engaging individuals to be an active part of the process.  So although surprising, I guess it is an indicator of how we have historically done business in education.  

What was one this that hurts your heart?

There was some tension or misconceptions between IT and faculty at times.  I wish as an organization, public servants, we could all take some time to perspective take and give folks the benefit of the doubt that they are doing the best they can.

Additional Resources:

Below you will find more details as to the project (see Appendix A) and the Summit event (see Appendix B).  Materials, including presentation slides, handouts, and notes, can be found on the wiki for the Summit at:

Sunday, May 4, 2014

Research: Social networking and social media for learning, communication, and satisfaction

Callaghan, N., & Bower, M. (2012).  Learning through social networking sites – the critical role of the teacher. Educational Media International, 49(1), 1-17.

deMarcos, L., Dominques, A., Saenz de Navarrete, J., & Pages, C. (2014).  An empirical study comparing gamification and social networking on e-learning. Computers & Education, 75, 82-91

Junco, R., Elavsky, C. M., Heiberger, G. (2012). Putting Twitter to the test: assessing outcomes for student collaboration, engagement, and success. British Journal of Educational Technology, 44(2), 273–287. doi:10.1111/j.1467-8535.2012.01284.x

Junco, R., Heiberger, G., & Loken, E. (2011). The effect of Twitter on college student engagement and grades. Journal of Computer Assisted Learning, 27(2), 119-132. doi:10.1111/j.1365-2729.2010.00387.x

Rahimah, C.W.I, Prain, V., & Collet, P. (2014). Perceived Learning Strategies of Malaysian University Students in Web 2.0-based English as a Second Language Informal Learning. GEMA Online Journal of Language Studies,14(1), 29-42.

Toetenel, L. (2014). Social networking: a collaborative open educational resource. Computer Assisted Language Learning. 27(2), 149-162.

Wednesday, April 30, 2014

Readiness checklist for online and blended programs

I am developing a checklist for institutions or units (school, colleges, or departments) that are thinking about launching an online degree program. Please share any additional items that should be on the checklist.

Align with university strategic plans
Align with academic program plan
Align with university/program technology plans
Identify market demand
Develop marketing strategies (via @mel1017)
Provide research and trends to support
Develop pricing structure
Fee in lieu of tuition
Student DE fee
Financial Compensation 
Faculty Stipend, Buyout, or Overload 
Establish Training and Development  
Faculty development 
Components of training and development  
Technical support
Technical training 
Technical production 
Pedagogical support and instructional strategies
Personalized (individualized) support based on need 
Non-Financial Compensation and Motivators 
Aspect of tenure and promotion 
Certificate of CV-Builder 
Award or public recognition
Working remotely

Friday, April 25, 2014

Teaching an online course: A resource list

Lehman, R. M., & Conceição, S. C. O. (2014). Motivating and Retaining Online Students: Research-Based Strategies That Work. San Francisco: Jossey-Bass.,miniSiteCd-JBHIGHERED.html

Joosten, T., (2012).  Social Media for Educators. San Francisco: Jossey-Bass.

Conceição, S. C., & Lehman, R. M. (2011). Managing Online Instructor Workload: Strategies for Finding Balance and Success. San Francisco: Jossey-Bass.,miniSiteCd-JBHIGHERED.html

Lehman, R. M., & Conceição, S. C. (2010). Creating a Sense of Presence in Online Teaching: How to ‘Be There’ for Distance Learners. San Francisco: Jossey-Bass.,miniSiteCd-JBHIGHERED.html

Conrad, R. & Donaldson, J. (2004). Engaging the Online Learner : Activities and Resources for Creative Instruction. San Francisco: Jossey-Bass.

Michaelsen, L.K., Knight, A.B., & Fink, L. D. (2004.) Team-based learning: A transformative use of small group in college teaching. Sterling, VA: Stylus Publishing.

Barkley, E., Major, C., Cross, P., Angelo, T. (2004). Collaborative Learning Techniques: A Handbook for College Faculty. San Francisco: Jossey-Bass.

Palloff, R.M. & Pratt, K., (2004). Collaborating Online : Learning Together in Community. San Francisco: Jossey-Bass.

Palloff, R.M. & Pratt, K., (2003). The Virtual Student: A Profile and Guide to Working with Online Learners. San Francisco: Jossey-Bass.

Wiggins, G. & McTighe, J. (2000). Understanding by Design. Englewood Cliffs, N.J: Prentice Hall

Palloff, R.M. & Pratt, K., (1999). Building Learning Communities in Cyberspace : Effective Strategies for the Online Classroom. San Francisco: Jossey-Bass.

Huba, J., & Freed, M. (1999). Learner Centered Assessment on College Campuses. Pearson.

Angelo, T. and Cross, P. (1993). Classroom Assessment Techniques : A Handbook for College Teachers. San Francisco: Jossey-Bass.

Walvoord, B., Anderson, V., Angelo, T. (1998). Effective Grading: A Tool for Learning and Assessment. San Francisco: Jossey-Bass.

Ambrose, How Learning Works via @gchinn

Ko, Teaching Online via @gchinn

Paloff and Pratt, The Excellent Online Instructor via @mel1017

And many below in the comments section from @jrhode

Tuesday, April 1, 2014

Ensuring quality and determining effectiveness: An ELI Focus Session Presentation


As many institutions have invested in faculty development programming and understand that it is pivotal to the success of innovation in course designs and academic programming, there is a need to ensure that the products resulting from these efforts are meeting institutional standards of quality for student learning and other outcomes. We have seen an array of mediated forms of learning (hybrid, blended, flipped, online, self-paced, competency-based, MOOCs, and more) being diffused across campuses and systems, and many of us have been asked to provide evidence of the effectiveness of our faculty development programming to ensure the quality of classes and programs. Administrators are looking for an ROI in faculty development as we are seeing decreases in funding, enrollments, and budgets. This presentation will share an approach to ensuring quality and evaluating the effectiveness of faculty development, including the sharing of resources.

Outcomes: Learn about a life cycle of ensuring quality in faculty development * Identify steps in a backward-design approach to evaluating the effectiveness of faculty development * Share potential resources to use in future efforts 

Link to presentation:




UWMLTC Faculty Development Evaluations:

Teaching Program:

Support Group:

Teaching Certification for Blended:

Lazirko Award

Peer eval handbook can be found at: A condensed version of the peer evaluation guide can be found at:

Course evaluation checklist:

Syllabus checklist:

Learning module checklist:

Assessment plan checklist:

ALN 2011, Online Programming Guide:

Tanya Joosten's Blog:

Tanya's Survey Wiki:

POD list of resources (on WikiPODia)

Van Note Chism, N., Holley, M., & Harris, C. (2012, July). Annotated Bibliography on Impact of Educational Development Interventions. Indiana University, Bloomington, IN.

From ELI Learning Circle, Rubrics/Metrics

“Chico model” : __
Quality Matters : __
Guskey’s five levels of prof development __>>
Also check out the next iteration of the (CSU-Cal State University) Chico Model which is being adopted by the whole 23-campus CSU system. This model is called “QOLT” Quality Online Learning & Teaching: __
Additional resources on ensuring quality in online and blended, including evaluating, please see:__

UCF Pedagogical Practice

ELI Faculty Engagement and Development Resource List

Thursday, March 13, 2014

Learning Technology Roadmap for UW System

What are you doing to plan for your learning technology future?

As a member of the Learn@UW Executive Committee and chair of the Academic Technology Roadmap Task force, we are trying to do some strategic directioning around the use and potential future needs of learning technologies.  I should note it is called the academic technology, blah, blah, but I really prefer the term learning technology when discussing technologies that have the potential when used appropriately to impact student learning.  I am very excited about this process and here is a little more about it:

"With the rapidly changing instructional environment and technology offerings, an academic roadmap is needed to guide the Learn@UW Executive Committee in making key decisions in planning and budget exercise, specifically:

Prioritize financial support for system wide instructional applications
Build capacity and responsiveness for future needs
Develop an understanding of the learning ecosystem within the system"

See for more details.

We have several things that we are planning to ensure quality in our process and product.  Now, I am not new to this having co-chaired UWM's digital future (teaching and learning track) planning (see  Also, I am currently participating in our Chancellor's campus planning, and for a couple years have been participating in State of Wisconsin DPI digital learning planning.

But, I want to know what you all are doing to plan for the future.

So, please share with me by responding to this post or putting on social media (@tjoosten).  I hope to put together a resource to help all of us in education plan for the future.