The rapidtide package

Rapidtide is a suite of Python programs used to model, characterize, visualize, and remove time varying, physiological blood signals from fMRI and fNIRS datasets. The primary workhorses of the package are the rapidtide program, which characterizes bulk blood flow, and happy, which focusses on the cardiac band.

Full documentation is at: http://rapidtide.readthedocs.io/en/latest/

|PyPi Latest Version| |PyPi - Python Versions| |License| |Documentation Status| |CircleCI| |Coverage| |DOI| |Funded by NIH|

The rapidtide program

Rapidtide is also the name of the first program in the package, which is used to perform rapid time delay analysis on functional imaging data to find time lagged correlations between the voxelwise time series and other time series, primarily in the LFO band.

Why do I want to know about time lagged correlations?

This comes out of work by our group (The Opto-Magnetic group at McLean
Hospital - http://www.nirs-fmri.net) looking at the correlations between
neuroimaging data (fMRI) and NIRS data recorded simultaneously, either
in the brain or the periphery. We found that a large fraction of the
\"noise\" we found at low frequency in fMRI data was due to real,
random[*] fluctuations of blood oxygenation and volume (both of which
affect the intensity of BOLD fMRI images) in the blood passing through
the brain. More interestingly, because these characteristics of blood
move with the blood itself, this gives you a way to determine blood
arrival time at any location in the brain. This is interesting in and of
itself, but also, this gives you a method for optimally modelling (and
removing) in band physiological noise from fMRI data (see references
below).

After working with this for several years we\'ve also found that you
don\'t need to used simultaneous NIRS to find this blood borne signal -
you can get it from blood rich BOLD voxels for example in the superior
sagittal sinus, or bootstrap it out of the global mean signal in the
BOLD data. You can also track exogenously applied waveforms, such as
hypercarbic and/or hyperoxic gas challenges to really boost your signal
to noise. So there are lots of times when you might want to do this type
of correlation analysis.

As an aside, some of these tools are just generally useful for looking
at correlations between timecourses from other sources – for example
doing PPI, or even some seed based analyses.

[*] \"random\" in this context means \"determined by something we don\'t
have any information about\" - maybe EtCO2 variation, or sympathetic
nervous system activity - so not really random.

Correlation analysis is easy - why use this package?
````````````````````````````````````````````````````

The simple answer is \"correlation analysis is easy, but using a
prewritten package that handles file I/O, filtering, resampling,
windowing, and the rest for you is even easier\". A slightly more complex
answer is that while correlation analysis is pretty easy to do, it\'s
hard to do right; there are lots and lots of ways to do it incorrectly.
Fortunately, I\'ve made most of those mistakes for you over the last 8
years, and corrected my code accordingly. So rather than repeat my
boring mistakes, why not make new, interesting mistakes? Explore your
own, unique chunk of wrongspace…


Happy
-----

More recently, inspired by Henning Voss\' paper on hypersampling of
cardiac signals in fMRI, we developed a method to extract and clean
cardiac waveforms from fMRI data, even when the fMRI TR is far too long
to properly sample cardiac waveforms. This cardiac waveform can then be
to track the pulsatile cardiac pressure wave through the brain in
somewhat the same way that we track the LFO signals. Among other things,
this allows you to get cardiac waveforms during scans even when either
1) you didn\'t use a plethysmograph, or 2) you did, but the recording was
of poor quality, which happens more than you might think.

What does \"happy\" have to do with any of this?
````````````````````````````````````````````````

As to why happy is part of rapidtide, that\'s partially for practical reasons (the
libraries in rapidtide have an awful lot of code that is reused in happy), and
partially thematically - rapidtide has evolved from a \"let\'s look at low
frequency signals in fMRI data\" package to a \"let\'s look at everything in
fMRI data EXCEPT neuronal activation\", so happy fits right in.


Why are you releasing this package?
===================================

For a number of reasons.

-  I want people to use it! I think if it were easier for people to do
   time delay analysis, they\'d be more likely to do it. I don\'t have
   enough time or people in my group to do every experiment that I think
   would be interesting, so I\'m hoping other people will, so I can read
   their papers and learn interesting things.
-  It\'s the right way to do science – I can say lots of things, but if
   nobody can replicate my results, nobody will believe it (we\'ve gotten
   that a lot, because some of the implications of what we\'ve seen in
   resting state data can be a little uncomfortable). We\'ve reached a
   stage in fMRI where getting from data to results involves a huge
   amount of processing, so part of confirming results involves being
   able to see how the data were processed. If you had to do everything
   from scratch, you\'d never even try to confirm anybody\'s results.
-  In any complicated processing scheme, it\'s quite possible (or in my
   case, likely) to make dumb mistakes, either coding errors or
   conceptual errors, and I almost certainly have made some (although
   hopefully the worst ones have been dealt with at this point). More
   users and more eyes on the code make it more likely that they will be
   found. As much as I\'m queasy about somebody potentially finding a
   mistake in my code, I\'d rather that they did so, so I can fix it[‡].
-  It\'s giving back to the community. I benefit from the generosity of a
   lot of authors who have made the open source tools I use for work and
   play, so I figure I can pony up too.

[‡] or better yet, you, empowered user, can fix it, and push back a fix
that benefits everybody…

Stability, etc.
===============
This is an evolving code base. I\'m constantly tinkering with it. That
said, now that I\'ve sent this off into to the world, I\'m being somewhat
more responsible about locking down stable release points. In between
releases, however, I\'ll be messing with things, although for the most
part this will be restricted to the dev branch.
**It\'s very possible that at any given time the dev branch will be very broken,
so stay away from it unless you have a good reason to be using it.**
I\'ve finally become a little more modern and started
adding automated testing, so as time goes by hopefully the \"in between\"
releases will be somewhat more reliable.  That said, my tests routinely fail, even
when things actually work.  Probably should deal with that. Check back often for exciting
new features and bug fixes!

Testing lanes
=============

The test suite now supports marker-based lanes so you can run fast checks
separately from heavier workflow tests.

-  **unit**: fast parser/math/small mocked tests
-  **slow**: heavier workflow-oriented tests

Using pytest directly:

-  ``pytest -m unit``
-  ``pytest -m slow``
-  ``pytest -m "not slow"``

Using the local test helper script:

-  ``cd rapidtide/tests``
-  ``./runlocaltest unit``
-  ``./runlocaltest notslow`` (or ``./runlocaltest pr``)
-  ``./runlocaltest slow``
-  ``./runlocaltest all``

Recommended CI split:

-  Fast lane (default PR checks): ``pytest -m "not slow"``
-  Slow lane (scheduled/nightly or dedicated job): ``pytest -m slow``

Current CircleCI lane mapping:

-  Fast lane jobs (``-m "not slow"``):
   ``test_py310``, ``test_py311_with_optional``, ``test_py311_with_coverage``,
   ``test_py312``, ``test_py313``, ``test_py314``
-  Slow lane job (``-m slow``):
   ``test_py311_slow``

Lane intent and expected runtime:

-  ``unit``: parser/math/mocked checks intended for very fast local feedback.
   Typical runtime is usually a few minutes or less, depending on hardware.
-  ``not slow``: primary PR lane. Covers unit tests plus non-heavy workflows while
   excluding long-running integration tests.
-  ``slow``: heavy integration-style tests (fullrun/simroundtrip and other marked
   workflow tests). This lane is intended for dedicated CI jobs.

Suggested pre-push workflow:

-  Before most commits: ``pytest -m unit``
-  Before opening a PR: ``pytest -m "not slow"``
-  Before release or in scheduled CI: ``pytest -m slow``

Python version compatibility
============================
Since I depend on a number of
packages that have dropped Python 2.x support, as of rapidtide 2.0, so did rapidtide. And given that I use fairly
modern constructs, I don't support anything prior to Python 3.9.  The current UPPER limit is 3.12, because
tensorflow (needed for happy) does not yet support 3.13 or later.  In 2025, I don't imagine
anybody is running rapidtide on a system that can't upgrade to a modern Python, but if you are,
as of version 1.9.0 the package is also available in a docker
container (fredericklab/rapidtide), which has everything nicely installed in
a fully configured Python 3 environment, so there\'s really no need for me continue 2.x
support.  So now it's f-strings all the way, kids!

Ok, I\'m sold. What\'s in here?
===============================

-  **rapidtide** - This is the heart of the package - this is the
   workhorse program that will determine the time lagged correlations
   between all the voxels in a NIFTI file and a temporal \"probe\"
   regressor (which can come from a number of places, including the data
   itself) - it rapidly determines time delays… There are a truly
   bewildering array of options, and just about everything can be