The best way to preprocess before and after splitting #4900

Consider the following problem.

I have a 4-shank Neuropixels probe. There is a large piece of noise that I want to remove by doing a common median reference. I believe (for whatever reasons) that it will be better to subtract the median value taken from ALL channels/shanks. After this, I want to split the recording per shank up, and apply other preprocessing steps. You might even want to do another common median reference per shank after the global one.

Question: what's the best way to proceed. And should SpikeInterface support some syntactic sugar to allow you to do this with pipelines etc?

Some possible answers, with examples, give that you have a recording which cab en split by group.

1. Do two preprocessing pipelines
   [{'name': 'bandpass_filter', 'kwargs': {'freq_max': 3000}}, {'name': 'common_reference', 'kwargs': {}}]

preprocessing_before_split = [{'name': 'common_reference'}]
preprocessing_after_split = [{'name': 'common_reference'}]
pp_rec = si.apply_preprocessing_pipeline(
    si.apply_preprocessing_pipeline(
        recording, preprocessing_before_split
    ).split_by('group')), 
preprocessing_after_split)

Good:

• We don't have to do anything
  Bad
• Bit awkward.

2. Make split_by a preprocessing step

preprocessing_pipeline = [{'name': 'common_reference'}, {'name', 'split_by', 'kwargs': {'property': 'group'}, {'name': 'common_reference'}]
pp_rec = si.apply_preprocessing_pipeline(recording, preprocessing_pipeline)

Good: simple for user
Bad:

• Typing gets more and more complex - Current if apply_preprocessing_pipeline takes in a rec/dict it returns a rec/dict. Now a rec might return a dict, if split_by is around.

Any other ideas?

If we stick with 1, let's document it. If we aim for 2, let's do it!

I think I prefer 1...