CUDA: larger SRAM reads for tile FA, AMD FP16 dot

[JohannesGaessler]

See https://github.com/iacopPBK/llama.cpp-gfx906 . AMD GPUs support reads of up to 16 bytes from SRAM. This PR extends the tile FlashAttention CUDA kernel with support for reads of 8 or 16 bytes. The FP32 -> FP16 type conversion is also done prior to writing the data to SRAM to reduce I/O further.

I also checked the AMD ISA documentation for v_dot2_f32_f16 support and adjusted the code paths accordingly; it seems to be available everywhere except for RDNA 1.

Performance changes

GPU	Model	Microbatch size	Test	t/s master	t/s 5bae9f9	Speedup
MI50	gemma 2B Q4_0	16	pp16384	629.54	716.54	1.14
MI50	gemma 2B Q4_0	32	pp16384	728.28	911.94	1.25
MI50	gemma 2B Q4_0	512	pp16384	1412.40	2141.40	1.52
MI50	llama 1B Q4_0	16	pp16384	927.48	998.83	1.08
MI50	llama 1B Q4_0	32	pp16384	1189.48	1338.73	1.13
MI50	llama 1B Q4_0	512	pp16384	2278.02	2898.30	1.27
MI50	llama 8B Q4_0	16	pp16384	277.56	294.46	1.06
MI50	llama 8B Q4_0	32	pp16384	334.59	366.55	1.10
MI50	llama 8B Q4_0	512	pp16384	508.27	606.21	1.19
RX 6800	gemma 2B Q4_0	16	pp16384	399.91	636.85	1.59
RX 6800	gemma 2B Q4_0	32	pp16384	313.91	992.28	3.16
RX 6800	gemma 2B Q4_0	512	pp16384	568.27	1897.49	3.34
RX 6800	llama 1B Q4_0	16	pp16384	644.87	903.19	1.40
RX 6800	llama 1B Q4_0	32	pp16384	658.72	1336.48	2.03
RX 6800	llama 1B Q4_0	512	pp16384	897.77	2301.67	2.56
RX 6800	llama 8B Q4_0	16	pp16384	172.06	234.54	1.36
RX 6800	llama 8B Q4_0	32	pp16384	174.27	328.20	1.88
RX 6800	llama 8B Q4_0	512	pp16384	231.13	530.74	2.30

[slaren]

Wouldn't this work?

Suggested change

	static_assert(nbytes == 0 && nbytes == -1, "bad nbytes");
	static_assert(false, "bad nbytes");

[JohannesGaessler]

I tried this first, it failed during the host pass.