prior 发现 cupy.cuda.cufft.CuFFTError: CUFFT_EXEC_FAILED 错误
/cc @Berrysoft 在 impreza0 和 ceres 上都观测到这个问题，这是服务器的问题吗？
Traceback (most recent call last):
File "/mnt/stage/xuchuang/JNE/bappen/FSMP/./fsmp_prior.py", line 39, in <module>
wave_deconv = multi_deconv(
^^^^^^^^^^^^^
File "/mnt/stage/xuchuang/JNE/bappen/FSMP/analysis/lucyddm.py", line 85, in multi_deconv
char = lucyddm(
^^^^^^^^
File "/mnt/stage/xuchuang/JNE/bappen/FSMP/analysis/lucyddm.py", line 40, in lucyddm
relative_blur = waveform / fftconvolve(wave_deconv, spe, mode="full", axes=1)[:, l_ser - 1 : l_psi]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/gentoo/usr/lib/python3.11/site-packages/cupyx/scipy/signal/_signaltools.py", line 185, in fftconvolve
out = _st_core._freq_domain_conv(in1, in2, axes, shape, calc_fast_len=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/gentoo/usr/lib/python3.11/site-packages/cupyx/scipy/signal/_signaltools_core.py", line 165, in _freq_domain_conv
sp1 = fftn(in1, fshape, axes=axes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/gentoo/usr/lib/python3.11/site-packages/cupyx/scipy/fft/_fft.py", line 468, in rfftn
return func(x, s, axes, norm, cufft.CUFFT_FORWARD, 'R2C',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/gentoo/usr/lib/python3.11/site-packages/cupy/fft/_fft.py", line 248, in _fft
a = _exec_fft(a, direction, value_type, norm, axes[-1], overwrite_x)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/gentoo/usr/lib/python3.11/site-packages/cupy/fft/_fft.py", line 191, in _exec_fft
plan.fft(a, out, direction)
File "cupy/cuda/cufft.pyx", line 500, in cupy.cuda.cufft.Plan1d.fft
File "cupy/cuda/cufft.pyx", line 520, in cupy.cuda.cufft.Plan1d._single_gpu_fft
File "cupy/cuda/cufft.pyx", line 1145, in cupy.cuda.cufft.execD2Z
File "cupy/cuda/cufft.pyx", line 169, in cupy.cuda.cufft.check_result
cupy.cuda.cufft.CuFFTError: CUFFT_EXEC_FAILED
srun: error: impreza0: task 0: Exited with exit code 1