Handling complex numbers

[ngergihun]

There is an increasing need for complex value data processing. It would be nice to discuss how to handle the complex numbers in quasar. There are several consideration about this:

Complex number creation:

• Inside the widgets with different input for amplitude/phase/real/imag parts?
• Inside the readers? (it only works if both parts of the complex data are in one file)
• Separate (Complexify/Decomplexify?) widget?
  • Input: user could choose if inputs were amp/phase or real/imag
  • Output: complex datatable

Processes/widgets:
Should widgets check if the input data is complex and pop an error if they are not compatible with complex value data?

Display/dicomplexify:
For displaying data and for some data processing user would need to separate the parts of complex values.
Users should be able to choose between abs/phase or real/imag as outputs

• Same widget as the complexity?

We are very curious about everyone's ideas regarding this topic.

[markotoplak]

Technical limitations: Table's .X and .Y are float64. So, for now, if we wanted complex numbers natively in a table, they could only go into .metas.

Please, provide specific use cases. These things are hard to do in general.

[stuart-cls]

Thanks for raising this! I ran into this with our dual-comb spectrometer a while back. I've also never been super happy with the complex handling in OWFFT that requires multiple-rows per measurement.

I tried the brute force approach on Marko's limitation: Redefine the .X array to complex128. My idea was to make a ComplexTable that would do something sensible (I think for that application I was just going to return the .real component, which is probably not what you want for SNOM). The code is from 2022 so probably has diverged, but basically I just allow defining a DTYPE class attribute: ComplexTable. I did not test if that code still works (or ever did).

So a specific use case for that system:

1. Reader returns a ComplexTable
2. Arithmetic like averaging, smoothing and (most important) calculating from a different reference (see example tests in gist ) works because (hopefully) those calculations don't make assumptions about the dtype and use the array implicitly
3. Other code casts the table to float64 implicitly (from .real) maybe with a warning
4. Probably need a decomplex widget (like other unique Table implementations) to cleanly return to Table as you suggest

Perhaps we can collect a few workflows and highlight where the complex special datastructure (whatever it is) would improve things. My feeling is that most cases it's something like [Data -> Processing] -> Analysis where the square brackets part is complex.

[markotoplak]

Perhaps we can collect a few workflows and highlight where the complex special datastructure (whatever it is) would improve things. My feeling is that most cases it's something like [Data -> Processing] -> Analysis where the square brackets part is complex.

I certainly like your approach of keeping a separate data structure for a part of the workflow. (That is why I asked for use cases.)

[stuart-cls]

I updated my previous ComplexTable to the latest Orange in #826 .