Building an Apache Airflow slurm pipeline for 3D keypoints and spikesorting

Posted on Thu 15 August 2024 in Machine Learning

Summary

[Analyzing large neuro-behavioral datasets ]

I first used Apache Airflow in 2015

Keypoints pipeline

[outline of how the keypoints pipeline works]

VGG network outline. (image source)

Training new networks

Airflow Implementation

[]

Dynamically updating Airflow

[]

Monitoring analysis timing

Debugging pipeline errors

In [ ]: