Supervised Machine Learning Readiness: A No- and Low-Code Learning Series

Current machine learning educational resources are often designed for the future machine learning developer and don't show examples relevant to the Earth. They also typically require a deep understanding of statistics, advanced math, and programming, which is inaccessible to the average learner. However, in the same way that there are currently more end-users of numerical models than developers of numerical models, we anticipate the same will hold for machine learning models. Given this, we sought to foster the savvy use of machine learning in the Earth systems sciences by teaching end-users to critically judge machine learning models and their outputs.

Scaffolding this skill requires a general understanding of how machine learning models are created. Through the use of schematic diagrams, structured processes, and learner-led inquiry, learners can still engage with machine learning model development without requiring extensive prerequisite knowledge of advanced math, statistics, or programming. To address this need, we developed Supervised Machine Learning Readiness: a no- and low-code learning series tailored for Earth systems scientists seeking to build machine learning literacy without prior experience.

About the Series

Supervised Machine Learning Readiness is a series of three, one-hour modules designed to introduce Earth systems scientists and students to the use of supervised machine learning (ML) in their field. Each module builds on the previous to reinforce concepts and encourage learners to experiment and test their understanding. The series was initially designed for a flipped classroom, allowing learners to complete Foundationsoutside of the classroom, and Applications and Analysis as lab projects. However, the flexibility of the modules allows for a variety of implementations that best suit the needs of the learners and the learning environment.

The series begins with Foundations, a no-code conceptual introduction to supervised ML in the Earth system sciences. Learners are introduced to Sam, a research assistant tasked with analyzing forecasts for winter weather precipitation type prediction. Throughout the module, learners build up a conceptual framework for how supervised ML models are developed. They see examples of problem framing, data handling, and the iterative process of developing a model from data.

The second module, Applications, is a Jupyter Notebook-based resource that continues the scenario from Foundations. Learners are asked to first recreate Sam's model, then make improvements to it based on their independent data exploration and expanded evaluation metrics. Despite this module being in a Python-based workspace, learners are not asked to write any code. Learners see code and execute cells, but they explore and design the ML model using no-code widgets. As they progress through the module, learners complete handbook exercises, which are guided short-answer summaries of their data exploration and model design choices they've made. A rubric for the handbook is available for learners to reference along the way.

The final module, Analysis, is also a Jupyter Notebook activity. In this module, learners are asked to analyze a new scenario modeled after a real incident. In 2024, an atmospheric observation station in western North Carolina was destroyed by Hurricane Helene. This module asks learners to design and test an ML model that could backfill the missing data at that location, given the data at nearby observation stations, and make a decision on whether this model is an appropriate solution to the scenario. In line with Applications, learners complete short-answer questions in a handbook document with an accompanying rubric. Analysis is designed for groups so that learners gain practice building consensus on an open-ended task.

Access the Series

The series is available for public use on NSF Unidata eLearning with a free account. Visit https://elearning.unidata.ucar.edu/course/view.php?id=13 to enroll for free. This course provides access to all resources and setup instructions. You can also see other details, including learning objectives and classroom implementation, on the Supervised Machine Learning Readiness page in the Teach the Earth catalog. These resources may be downloaded and adapted for your specific needs.

Classroom Implementation and Assessment

The series was piloted in a Climatology course at Metropolitan State University of Denver in Spring 2025. As a part of the Climate Change minor curriculum, this course included around twenty students from a broad range of majors and academic levels. The modules were offered over a series of three weeks, each as lab activities. We assessed student learning with a multifaceted approach. First, students were given a pre-test before beginning Foundations and took the same test again after completing the module. Students also completed the Applications and Analysis handbooks, which were graded with the same rubric that was provided to them at the time of the lab.

Pre- and post-test scores increased from a class average of 42% to 73% following completion of Foundations. The Applications handbooks were completed as an individual activity and proved to be more challenging to students, with an average score of 11.3 out of 20 possible points. The Analysis handbooks were completed in small groups and showed a much higher average score of 20.9 out of 25 possible points.

Students were also asked to complete a two-part reflection activity three weeks after completing the final module. The first reflection question asked the students to apply their knowledge of machine learning to a hypothetical scenario: asking a presenter at a conference questions about their use of machine learning in their research and providing sample responses. From their collective responses, we identified key themes regarding model accuracy and data choice.

1. Learners know how to inquire about the accuracy and evaluation metrics used to determine a model's utility, for example, "How did you evaluate the accuracy of the model's hail size prediction? We evaluated accuracy using root mean squared error (RMSE) and correlation coefficient (R²)."
2. Learners inquire about the choice of data used as input to a model, including questions regarding the source, amount, and quality of data, for example, "Did you look to see if there were notable errors before gathering data? Yes, we verified that each tower was working properly."

The second included a portion of a real job description and asked students to map the skills they learned in the series to the job responsibilities in the listing.

1. Learners recognize that understanding machine learning is a useful skill for potential jobs in the field of meteorology, as it demonstrates their ability to use new tools to deliver results (weather prediction and forecasting). For example, "These skills will enable me to synthesize accurate forecasts and apply new forecasting tools in an operational environment."
2. Data analysis and interpretation: Learners connected the data analysis skills they practiced in the modules with desirable job skills, and recognized that the short-answer responses they completed were a practice in communicating data and results. For example, "I gained experience interpreting model output and communicating uncertainty."

Lessons Learned

In the pilot, we found that this series was most appropriate for upper-level undergraduate students, particularly those who had prior coursework in graph interpretation and tabular data (spreadsheet) exploration. For audiences that may have novice-level graph interpretation or data exploration skills, we recommend using the Applications and Analysis modules as full-class (for small class sizes) or large group (10-15) activities. This method may allow for more guided instruction and redirection, particularly when completing the evaluation exercises in the handbooks.

Our pilot demonstrated significant improvement in student knowledge, with pre- to post-test scores improving from 42% to 73%. Additionally, students working in small groups (Module 03) achieved higher scores than those working individually (Module 02). However, these findings require caution, as the activities involved different modules with varying complexity and assessment criteria, complicating direct comparisons. Further research is needed to isolate the specific effects of collaborative learning on immediate and long-term learning outcomes.

Conclusion

Supervised Machine Learning Readiness is a no- and low-code series designed to make machine learning literacy more accessible to a broader range of learners. These resources are free to access and adapt on NSF Unidata eLearning, and are ready to use in a variety of upper-level undergraduate settings and beyond. Whether you are interested in teaching these skills or learning them yourself, Supervised Machine Learning Readiness sets a solid foundation for savvy machine learning usage and continued studies.

Acknowledgements 

We thank the NOAA National Severe Storms Laboratory for contributing mPing data, NCAR MILES for data processing and support, and the North Carolina State Climate Office for contributing NC ECONet data and media to this project.

This work was supported by NSF Unidata under award #2319979 from the US National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

References

Haupt, S. E., Gagne, D. J., Hsieh, W. W., Krasnopolsky, V., McGovern, A., Marzban, C., Moninger, W., Lakshmanan, V., Tissot, P., & Williams, J. K. (2022). The History and Practice of AI in the Environmental Sciences. Bulletin of the American Meteorological Society, 103(5), E1351-E1370. https://doi.org/10.1175/BAMS-D-20-0234.1

Authors

^aNSF Unidata, University Corporation for Atmospheric Research, Boulder, Colorado, USA, https://orcid.org/0000-0002-0956-6401 and https://orcid.org/0000-0002-4171-0004

^bMetropolitan State University of Denver, Denver, Colorado, USA, https://orcid.org/0000-0001-6852-3375