Tutoring Analytics Framework
A Structured Three-Tiered Methodology for Program Evaluation, Econometrics & ROI
Author: Geoffrey Bailey (2023; rev. 2025, 2026)
Published By: International College Learning Center Association (ICLCA)
Purpose: Transitioning learning centers from descriptive tracking to econometric modeling, quasi-experimental research, and financial ROI evaluation.
Framework Overview
The Tutoring Analytics Framework provides higher education institutions with a comprehensive roadmap to measure and evaluate the impact of learning support services. It guides centers progressively through three analytical levels:
Foundational Descriptive Baseline
Establishes baseline operational tracking to ensure reporting compliance, resource allocation, and basic program accountability. Key metrics include:
- 1.1 Student Visits Tracking: By visit type, location, subject, professor, college/school, tutor, etc.
- 1.2 Student Hours Logged: By type of service, modality, duration, etc.
- 1.3 Utilization Trends: Peak times, wait-times, no-show and cancelation rates.
- 1.4 Courses & Subjects Served: Active inventory aligned with gateway priorities.
- 1.5 D/F/W Rates: Tracked by course, subject, and academic discipline.
- 1.6 Unfilled Tutoring Requests: Logged by course/term to predict demand and hiring needs.
- 1.7 Longitudinal Participation Trends: Semester-to-semester and year-to-year volume growth.
- 1.8 Participant Course Grades & GPAs: Disaggregated by class section (e.g., BIOL 101) and discipline.
- 1.9 Student Demographic Utilization: Race/ethnicity, sex, first-generation status, and Pell grant eligibility.
- 1.10 Tutor Staff Diversity & Demographics: Staff profile evaluation for inclusive representation.
- 1.11 Survey Data & Testimonials: Quantitative feedback and qualitative themes comparing participants vs. non-participants.
Disaggregated Outcomes & Impact
Moves beyond raw numbers to compare participant vs. non-participant outcomes, determine intervention efficacy, and measure cognitive/affective growth. Key metrics include:
- 2.1 Participant vs. Non-Participant Grade Analysis: Aggregate comparisons by course, support type, sex, ethnicity, first-gen, and Pell status.
- 2.2 Student Population Sub-Group Comparisons: Usage and performance across entry cohorts (HS GPA bands, test scores, residency).
- 2.3 Support Modality Performance Analysis: In-person vs. online, individual vs. group, PAL/SI, drop-in vs. scheduled.
- 2.4 Retention & Graduation Rate Comparisons: Multi-term persistence and 4/6-year graduation differentials.
- 2.5 Transitional Coursework Success Rates: Developmental and co-requisite intervention pass rates disaggregated by population.
- 2.6 Tutor Staff Cognitive & Affective Growth: Personal, professional, and leadership development of student tutors.
- 2.7 Dosage Threshold Benchmarking: Minimum tutoring contact hours needed for statistically significant grade gains.
- 2.8 Cohort Penetration Rate & Impact: Proportion of high-need target populations actively served.
- 2.9 Academic Progression & Credit Velocity: Earned credit hours and degree milestone progression.
- 2.10 Early Alert Conversion Rate Analysis: Sequential logistic regression evaluating early alert student referrals.
- 2.11 Direct Assessment of Learning Outcomes: Cognitive and metacognitive skill gains, concept mastery, and problem-solving confidence.
Causal Inference, Predictive Analytics & Financial Modeling
Applies econometric modeling, quasi-experimental controls, predictive analytics, and financial ROI evaluation. Key metrics include:
- 3.1 Systemic Equity Analysis: Historically marginalized population performance compared to white student baselines.
- 3.2 Marginalized Participant Cohort Performance: Low-income/Pell, underserved, and first-gen participant outcomes vs. non-participants.
- 3.3 Course Failure Pattern Clustering: Identifying systemic course failure sequences and co-enrollment bottleneck pairs.
- 3.4 English as an Additional Language (EAL) Performance: Pass rates, language acquisition, and persistence for EAL participants.
- 3.5 Cost-Effectiveness & Retained Tuition Revenue: Cost per participant and net tuition dollars preserved through retention.
- 3.6 Predictive Outreach & Decision-Tree Modeling: Identifying student sub-profiles most likely to respond to targeted outreach.
- 3.7 Embedded Tutoring Impact Assessment: Gateway active-learning courses compared to traditional lecture sections.
- 3.8 Propensity Score Matching (PSM): Quasi-experimental matching to control for self-selection bias and isolate true treatment effect.
- 3.9 Repeat-Attempt Control & ANCOVA Modeling: Evaluating course repeaters utilizing tutoring vs. matched non-participant repeaters.
- 3.10 Time-Series ARIMA Supply/Demand Forecasting: Modeling hourly and term demand trends to optimize staffing and budgets.
- 3.11 Relative Course Grade Modeling: Controlling for covariates (GPA, test scores, demographics) to measure impact in rigorous STEM courses.
Implementation Template Architecture
The three fillable institutional template documents provide a standardized, 4-part operationalization matrix for every metric across all three levels:
1. Field & Institutional Purpose
Documents specific campus use cases, target sample populations, and key operational questions.
2. Formula & Methodology
Defines exact mathematical equations, variable definitions, and step-by-step statistical methods.
3. Resources & Software Links
Houses URLs to statistical scripts (R, Python, SPSS, Stata), management software (TutorTrac, Penji, Accudemia), and IR dashboards.
4. Response to Data Findings
Details data interpretation, service operational adjustments, and narrative strategies for campus stakeholders.