research software engineering with python ucl

Breadcrumbs Section. Click here to navigate to respective pages.

Research Software Engineering with Python

DOI link for Research Software Engineering with Python

Get Citation

Writing and running software is now as much a part of science as telescopes and test tubes, but most researchers are never taught how to do either well. As a result, it takes them longer to accomplish simple tasks than it should, and it is harder for them to share their work with others than it needs to be.

This book introduces the concepts, tools, and skills that researchers need to get more done in less time and with less pain. Based on the practical experiences of its authors, who collectively have spent several decades teaching software skills to scientists, it covers everything graduate-level researchers need to automate their workflows, collaborate with colleagues, ensure that their results are trustworthy, and publish what they have built so that others can build on it. The book assumes only a basic knowledge of Python as a starting point, and shows readers how it, the Unix shell, Git, Make, and related tools can give them more time to focus on the research they actually want to do.

Research Software Engineering with Python can be used as the main text in a one-semester course or for self-guided study. A running example shows how to organize a small research project step by step; over a hundred exercises give readers a chance to practice these skills themselves, while a glossary defining over two hundred terms will help readers find their way through the terminology. All of the material can be re-used under a Creative Commons license, and all royalties from sales of the book will be donated to The Carpentries, an organization that teaches foundational coding and data science skills to researchers worldwide.

TABLE OF CONTENTS

Chapter | 6  pages, chapter 1 | 6  pages, getting started, chapter 2 | 32  pages, the basics of the unix shell, chapter 3 | 24  pages, building tools with the unix shell, chapter 4 | 26  pages, going further with the unix shell, chapter 5 | 26  pages, building command-line tools with python, chapter 6 | 36  pages, using git at the command line, chapter 7 | 42  pages, going further with git, chapter 8 | 32  pages, working in teams, chapter 9 | 26  pages, automating analyses with make, chapter 10 | 14  pages, configuring programs, chapter 11 | 28  pages, testing software, chapter 12 | 22  pages, handling errors, chapter 13 | 14  pages, tracking provenance, chapter 14 | 34  pages, creating packages with python, chapter 15 | 2  pages.

• Privacy Policy
• Terms & Conditions
• Cookie Policy
• Taylor & Francis Online
• Taylor & Francis Group
• Students/Researchers
• Librarians/Institutions

Connect with us

Registered in England & Wales No. 3099067 5 Howick Place | London | SW1P 1WG © 2024 Informa UK Limited

Research Software Engineering with Python

Download notes as PDF .

Powered by Jupyter Book .

• suggest edit

1. Introduction to Python

1. introduction to python #.

Why use scripting languages?

Python and the Jupyter notebook

Variables, using functions, and types

Loops and control

Data structures: lists, dictionaries, and sets.

1.0 Introduction to Python (10 minutes)

1.1 Variables (20 minutes)

1.2 Functions (20 minutes)

1.3 Types (20 minutes)

1.4 Containers (10 minutes)

1.5 Dictionaries (10 minutes)

1.6 Data Structures (5 minutes)

1.7 Control and Flow (15 minutes)

1.8 Iteration (10 minutes)

Total time: 2 hrs

Exercises #

Classroom exercises are grouped together at the end of the module: 1.9 Classroom Exercises . Each exercise is labelled with any sections whose contents are relevant. We recommend that instructors schedule the exercises to be done in groups during breaks in the taught content. However, it is important that participants also have some time away from their screens. Exercises can also be left as self-paced homework assignments if preferred.

• Prerequisites
• Experiences
• Assessments

Learning Outcomes

Version control learning outcomes.

By the end of this module students should be able to:

• Understand and communicate the benefits of using version control in the research domain
• Understand the workflow of adding, committing and pushing revisions in git
• Work collaboratively with git, both via pull requests and forks
• Describe the utility of branches, and set up projects where work is not solely contained in the master branch
• Use GitHub pages to create simple webpages using git and markdown
• Understand the differences between rebase and merge

Referencing modules: Version Control

Testing Learning Outcomes

• Understand and communicate the benefits of testing code in the research domain
• Explain the differences between unit, integration, component and regression testing. As well as the need for both positive and negative tests
• Discuss the benefits of using test frameworks
• Design test cases conceptually and implement these tests using PyTest
• Write tests which handle floating point values, to a domain appropriate tolerance level
• Use mocks to abstract remote resources using PyTest
• Use a debugger to identify problems in research code

Referencing modules: Testing your Code

Software Projects Learning Outcomes

• Understand and communicate the benefits of libraries to a research programmer
• Evaluate the quality of third-party libraries
• Install python packages using pip, conda and other sources
• Organise python code into a structured package
• Use argparse to handle command line arguments to executable scripts
• Understand and communicate the value of documentation and use appropriate tools to create documentation
• Discuss the pros and cons of different software design and development methodologies
• Understand software licenses, including how to select an appropriate language for a given project

Referencing modules: Software Projects

Construction and Design Learning Outcomes

• Understand and communicate the difference between software design and construction
• Understand the benefits of coding conventions and the importance of structured code for reproducible research
• Refactor code systematically, employing Object Orientated design principles
• Understand and apply common software design patterns in the research context
• Understand and discuss the use of polymorphism and inheritance in Object Orientated software design

Referencing modules: Construction and Design

Advanced Programming Techniques Learning Outcomes

• Understand functional programming techniques, and how to apply them to write clear, concise research software
• Understand how iterators and generators can be used to handle large datasets
• Understand the use of exceptions in Python and how to design software using exceptions
• Apply metaprogramming techniques to reduce repetition in code
• Understand the limits of metaprogramming

Referencing modules: Advanced Programming Techniques

Programming for Speed Learning Outcomes

• Understand and communicate the challenges and benefits of optimising research code
• Use numpy to optimise code
• Use cython to optimise code
• Understand the impact on performance differing data structures can have, and explain these impacts using scaling laws

Referencing modules: Programming for Speed

• ReadingLists@UCL Help

Browse Hierarchy COMP0233: COMP0233: Research Software Engineering with Python

Lists linked to comp0233: research software engineering with python.

Add list to this Module

Add existing node.

UCL Institute of Health Informatics

Software Development with Python for Health Data Science

In this course, students with a limited knowledge of programming will learn how to construct reliable, readable, efficient software for data-intensive research in a collaborative environment. The emphasis is on practical techniques, tips, and technologies to effectively build and maintain complex code. This is a intensive, practical course.

Module code

Ucl credits, course length, face to face dates.

Term 1, Weeks 6-10 & 12-16, Thursday 10:00 - 17:30

Assessment Dates

Module organisers.

Dr Holger Kunz .  Please direct queries to  [email protected]

• To equip students with limited experience of programming with the knowledge and skills required for the efficient development of useful and reliable software.
• To give students a detailed knowledge of programming techniques relevant to data science.
• To promote a professional approach to the design, implementation and testing of software.

Teaching and learning methods

Students will attend a lecture each week followed by a problem class. They will be supported by a comprehensive range of online materials developed for the programme by Research Software team. These include videos, moodle pages, online exercises, code notebooks.

Each week students will complete a range of programming assignments under the supervision of a teaching assistant. After each topic there is an online quiz, which allows students to assess their understanding of the material.

There will be one formative programming assignment which students will submit and on which they will receive formal feedback. This will be designed to help prepare students for the final summative assessment.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

MPHYM001/G001 - Research Software Engineering with Python Coursework II (UCL, 2017-18)

SSabas/ucl-softeng-cw2

Folders and files.

Repository files navigation

Refactoring trees, an exercise for a course in research software development in python, at ucl..

In this exercise, we will convert badly written code, provided here, into better-written code. We will do this not through simply writing better code, but by taking a refactoring approach, as discussed in the lectures. As such, we use of git version control, to make a commit after each step of the refactoring, with a commit message which indicates the refactoring you took, will be critical to success.

• Python 100.0%