sac2c issues
https://gitlab.sac-home.org/sac-group/sac2c/-/issues
2024-03-22T14:51:48Z
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2396
Distributed memory backend
2024-03-22T14:51:48Z
Thomas Koopman
Distributed memory backend
Reimplement the distributed memory backend based on Shray to ensure memory-scalability, and potentially greater performance.
## Multithreaded + distributed status:
The ground works is there: all the parallelism is created by restrictin...
Reimplement the distributed memory backend based on Shray to ensure memory-scalability, and potentially greater performance.
## Multithreaded + distributed status:
The ground works is there: all the parallelism is created by restricting the bounds and in principle orthogonal as to how these restricted with-loops are computed. However, the build system needs to be adjusted and probably some work needs to go into proper initialization.
## Side-effects
A function is assumed to have side-effects if it is an external function consuming a unique object, or a user-defined function type casting a unique object in its body. In that case, the function is computed only on the source node, and return arguments are broadcast. For now, we also consider functions that take or return hidden objects as having side-effects, though this can be relaxed in the future.
## Performance and correctness
This branch should behave correctly for all inputs as we fall back to replicated execution for constructs we cannot parallelize. Performance is outside of the scope of this merge request.
## Dependencies
This branch will only build if `gasnet` is installed, and environment variable `GASNet_ROOT` has been set to the installation. Shray needs the MPI or UDP backend of GASNet, so one of these dependencies also needs to be installed.
## TODO
- [x] implement reshape
- [x] implement multi-operator with-loops
- [x] fix hard-coded nametags in stdlib
- [x] nbody-naive
- [ ] FlashAttention alg 1
- [x] FlashAttention modified algorithm
- [ ] VolCalib
- [x] MG: multigrid method
- [ ] MG: initalisation (quickselect, RNG, some other funsies)
- [ ] Adjust CI to deal with GASNet
Thomas Koopman
Thomas Koopman
2024-04-21
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2395
Loop lifting fails
2024-03-13T15:11:10Z
Thomas Koopman
Loop lifting fails
Commit c0ba0c984b00f4109a3edb95597a517613c9f5c1
Consider the following program. `matmulT` only has to be computed `N / Br` times.
```
use Array: all;
#define d 128
#define N 1024
#define Br 128
#define Bc 256
noinline float[n:shp] i...
Commit c0ba0c984b00f4109a3edb95597a517613c9f5c1
Consider the following program. `matmulT` only has to be computed `N / Br` times.
```
use Array: all;
#define d 128
#define N 1024
#define Br 128
#define Bc 256
noinline float[n:shp] id(float[n:shp] x) { return x; }
noinline
float[m, n] matmulT(float[m, k] A, float[n, k] B)
{
return {iv -> tof(0) | iv < [m, n]};
}
inline
float FlashAttention(float[N, d] Q, float[N, d] K, float[N, d] V)
{
Qb = reshape([N / Br, Br, d], Q);
Kb = reshape([N / Bc, Bc, d], K);
O = tof(0);
for (j = 0; j < N / Bc; j++) {
Pj = {[i, a] -> matmulT(Qb[i], Kb[j])[a]
| [i, a] < [N / Br, Br]};
O += sum(Pj);
}
return O;
}
int main()
{
Q = id({iv -> tof(1) | iv < [N, d]});
K = id({iv -> tof(1) | iv < [N, d]});
V = id({iv -> tof(1) | iv < [N, d]});
O = FlashAttention(Q, K, V);
return _toi_S_(O);
}
```
However, the optimised code gives
```
/* Partn */
([ 0, 0 ] <= _flat_82=[i, a] (IDXS:_wlidx_920_Pj) < [ 8, 128 ] genwidth [ 8, 128 ])
{
_ivesli_930 = _idxs2offset_( [ 8, 128, 128 ], i, _iveras_1039, _iveras_1040);
_flat_84 = with /** FOLDABLE (all gen's const) **/
/** REFERENCED: 1 (total num refs) **/
{
/* Partn */
([ 0, 0 ] <= _pinl_540_iv=[_pinl_543__eat_146, _pinl_542__eat_145] (IDXS:_wlidx_921__flat_84) < [ 128, 128 ] genwidth [ 128, 128 ])
{
_ivesli_932 = _idxs2offset_( [ 8, 128, 128 ], _iveras_1041, _pinl_543__eat_146, _pinl_542__eat_145);
_ivesli_933 = _add_SxS_( _ivesli_930, _ivesli_932);
_pinl_538__flat_396 = _idx_sel_( _ivesli_933, Qb);
} : _pinl_538__flat_396 ;
} :
genarray( [ 128, 128 ], _pinl_450__flat_393, IDX(_wlidx_921__flat_84));
_flat_83 = _MAIN::matmulT( _flat_84, _flat_56) ;
```
computing it `N / Br * Br` times. The `[i, a]` loop should have been split up in an `[i]` and `[a]` loop, the `matmulT` lifted out of the `[a]` loop, and then inside of the `[a]` loop a suballoc can be done.
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2394
TCcountArgs internal compiler error
2024-03-11T09:15:20Z
Thomas Koopman
TCcountArgs internal compiler error
Following program fails with `Assertion "TCcountArgs (FUNDEF_ARGS (fundef)) == TYgetProductSize (arg_ts)" failed at /home/thomas/repos/sac2c/src/libsac2c/typecheck/specialize.c:321 -- UpdateFixSignature called with incompatible no of arg...
Following program fails with `Assertion "TCcountArgs (FUNDEF_ARGS (fundef)) == TYgetProductSize (arg_ts)" failed at /home/thomas/repos/sac2c/src/libsac2c/typecheck/specialize.c:321 -- UpdateFixSignature called with incompatible no of arguments!`
When removing the unused `softmax(float[m] x)`, the error does not occur, so probably something to do with function overloading.
```
use StdIO: all;
inline
float max(float a, float b)
{
return _ge_SxS_(a, b) ? a : b;
}
inline
float maximum(float[m] x)
{
return with {
([0] <= iv < [m]): _sel_VxA_(iv, x);
}: fold(max, _sel_VxA_([0], x));
}
inline
float[m] maximum(float[m, n] x)
{
return {iv -> _sel_VxA_([0], _sel_VxA_(iv, x))};
}
inline
float[m] softmax(float[m] x)
{
return x;
}
inline
float[m, n] softmax(float[m, n] x)
{
m = maximum(x);
print(_shape_A_(m));
return x;
}
int main()
{
m = 1000;
x = {iv -> 1f | iv < [m]};
StdIO::print(softmax(x));
return _toi_S_(_sel_VxA_([0], softmax(x)));
}
```
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2393
Out of memory compile error
2024-03-25T14:34:38Z
Thomas Koopman
Out of memory compile error
```
use StdIO: all;
int main()
{
m = 1000;
x = {iv -> 1 | iv < [m]};
StdIO::print(x);
return 0;
}
```
fails with `Out of memory: 751605325050240 bytes requested`. Compiled with commit `9b80bcfd0ad24d93a1cab9c365ac1fea723fe6d5...
```
use StdIO: all;
int main()
{
m = 1000;
x = {iv -> 1 | iv < [m]};
StdIO::print(x);
return 0;
}
```
fails with `Out of memory: 751605325050240 bytes requested`. Compiled with commit `9b80bcfd0ad24d93a1cab9c365ac1fea723fe6d5`, no flags
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2392
Bug in class implementation
2024-03-09T10:32:01Z
Artem Shinkarov
Bug in class implementation
Here is a definition of the class which is a counter, where the update effect is merged into the `value` accessor:
```c
class cnt;
classtype int;
export all;
cnt new (int x) {
return to_cnt (x);
}
int value (cnt &x) {
v = from_cnt ...
Here is a definition of the class which is a counter, where the update effect is merged into the `value` accessor:
```c
class cnt;
classtype int;
export all;
cnt new (int x) {
return to_cnt (x);
}
int value (cnt &x) {
v = from_cnt (x);
x = to_cnt (_add_SxS_ (v, 1));
return v;
}
```
Here is the file that uses this class:
```c
use cnt:all;
int main () {
a = new (1);
return value (a);
}
```
When I run this, I get the value 2 back, but the correct result is 1.
The version of sac2c I used is: `sac2c 1.3.3-MijasCosta-1161-gb543c`
Sven-Bodo Scholz
Sven-Bodo Scholz
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2391
Non-negativity constraint for type patterns
2024-03-08T10:44:22Z
Jordy Aaldering
Non-negativity constraint for type patterns
```
foo(int[o:oshp,i:ishp] a, int o)
```
In this example we need to ensure that `o` is not negative.
I suspect this is currently not enforced, and probably something horrible happens if `o` is negative.
```
foo(int[o:oshp,i:ishp] a, int o)
```
In this example we need to ensure that `o` is not negative.
I suspect this is currently not enforced, and probably something horrible happens if `o` is negative.
Jordy Aaldering
Jordy Aaldering
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2390
Towards a more lazy semantics
2024-03-07T17:31:30Z
Sven-Bodo Scholz
Towards a more lazy semantics
Several optimisations do perform redices that in a strict evaluation would never be executed.
A simple example is: `K (2, 3/0)`. Under a strict evaluation scheme it should not terminate / yield bottom; if we optimise this into `2`, this ...
Several optimisations do perform redices that in a strict evaluation would never be executed.
A simple example is: `K (2, 3/0)`. Under a strict evaluation scheme it should not terminate / yield bottom; if we optimise this into `2`, this changes the semantics even though, under a normal order regime, this is fine.
Quite some discussion on this matter was done in the context of the merge request for URR:
https://gitlab.sac-home.org/sac-group/sac2c/-/merge_requests/285#note_13840
The question remains: how do we want to deal with this?
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2389
UTCatenate crash in IVEXP with -doawlf, due to unexpected guard primitive
2024-03-06T14:40:37Z
Robert Bernecky
UTCatenate crash in IVEXP with -doawlf, due to unexpected guard primitive
N_type found where 31:N_id was expected. This is a recent failure,
likely due to a change in pattern-matching semantics.
[sacbug.sac](/uploads/368505b6c3871d061ad2e47ef51282c1/sacbug.sac)
N_type found where 31:N_id was expected. This is a recent failure,
likely due to a change in pattern-matching semantics.
[sacbug.sac](/uploads/368505b6c3871d061ad2e47ef51282c1/sacbug.sac)
Jordy Aaldering
Jordy Aaldering
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2388
1 / (1 + inf) gives nan instead of 0
2024-02-21T16:00:28Z
Thomas Koopman
1 / (1 + inf) gives nan instead of 0
# SaC version
sac2c 1.3.3-MijasCosta-1161-gb543c
build-type: DEBUG
built-by: "thomas" at 2024-02-21T12:07:11
# Test program
```
/* We need expf so Stdlib is necessary */
use Array: all;
use Math: all;
inline
float[*] logistics(float[...
# SaC version
sac2c 1.3.3-MijasCosta-1161-gb543c
build-type: DEBUG
built-by: "thomas" at 2024-02-21T12:07:11
# Test program
```
/* We need expf so Stdlib is necessary */
use Array: all;
use Math: all;
inline
float[*] logistics(float[*] x)
{
return {iv -> 1f / (1f + exp(-x[iv]))};
}
noinline
float[d:shp] id(float[d:shp] x)
{
return x;
}
int main()
{
c11 = id(with {}: genarray([6, 24, 24], -1875001.750000f));
/* Note that this is well defined under IEEE-754:
exp(-x[iv]) = +inf
1 + +inf = +inf
1 / +inf = 0+
*/
c1 = logistics(c11);
StdIO::print(c1);
return 0;
}
```
# Output
When compiling without arguments, prints all NaN (with gcc 13.2.0 and clang 14.0.6). When passing `-Xc -O2` it does give the correct result, which is all `0`s.
Stepping through with `gdb` shows that the `+ 1` and division are optimised away from O3 onwards. It is possible that the generated C code is incorrect, but `-fsanitize=undefined` finds nothing. I may have to boil this down to a minimal example and file a bug with `gcc` and `clang` if it is not on our end.
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2387
Missing default expression in WL
2024-02-19T23:35:10Z
Robert Bernecky
Missing default expression in WL
[UTReduce.sac](/uploads/6df9a4e53e68f7a0e8a8fa26321ec509/UTReduce.sac)
```
sac2c_d UTReduce.sac -doawlf -v1
Warning:
AWLF is enabled: -ecc enabled.
Warning:
AWLF is enabled: -extrema enabled.
Warning:
AWLF is enabled: -maxoptc...
[UTReduce.sac](/uploads/6df9a4e53e68f7a0e8a8fa26321ec509/UTReduce.sac)
```
sac2c_d UTReduce.sac -doawlf -v1
Warning:
AWLF is enabled: -ecc enabled.
Warning:
AWLF is enabled: -extrema enabled.
Warning:
AWLF is enabled: -maxoptcyc=20
//home/sac/sac/BASE/Stdlib/build/src-mt_pth/structures/ArrayBasics.sac:223: abort:
Genarray with-loop with missing default
expression found. Unfortunately, a default expression is necessary here to compute the shape of the result
Compilation failed while Transforming with-loop representation, 3 warning(s).
apex@medusa:/tmp/crud$
apex@medusa:/tmp/crud$ sac2c_d -V
sac2c 1.3.3-MijasCosta-1157-g02676
build-type: DEBUG
built-by: "sac" at 2024-02-14T10:15:11
```
Sven-Bodo Scholz
Sven-Bodo Scholz
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2386
crash in LUR if -doawlf
2024-02-14T16:33:30Z
Robert Bernecky
crash in LUR if -doawlf
```
Applying loop unrolling ...
TRAVERSE ERROR: node of type 50:N_type found where 31:N_id was expected.
apex@medusa:/tmp/crud$ sac2c -V
sac2c 1.3.3-MijasCosta-1157-g02676
build-type: DEBUG
built-by: "sac" at 2024-02-14T10:15...
```
Applying loop unrolling ...
TRAVERSE ERROR: node of type 50:N_type found where 31:N_id was expected.
apex@medusa:/tmp/crud$ sac2c -V
sac2c 1.3.3-MijasCosta-1157-g02676
build-type: DEBUG
built-by: "sac" at 2024-02-14T10:15:11
apex@medusa:/tmp/crud$ sac2c_d UTIndexSet.sac -doawlf -v4
```
[UTIndexSet.sac](/uploads/51520ffb42170b630db5394a7dc8b588/UTIndexSet.sac)
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2385
UTReduce.sac dies in WLF, if compiled with -maxwlur 1 -maxspec 0
2024-02-14T16:34:05Z
Robert Bernecky
UTReduce.sac dies in WLF, if compiled with -maxwlur 1 -maxspec 0
Title says it all
[UTReduce.sac](/uploads/d5d6523dbc3638bf08e04783d395196e/UTReduce.sac)
sac2c_d UTReduce.sac -maxwlur 1 -maxspec 0 -v4
Title says it all
[UTReduce.sac](/uploads/d5d6523dbc3638bf08e04783d395196e/UTReduce.sac)
sac2c_d UTReduce.sac -maxwlur 1 -maxspec 0 -v4
Sven-Bodo Scholz
Sven-Bodo Scholz
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2384
conflicting types, likely caused by maxwlur 1 and/or maxspec 0
2024-02-14T16:17:16Z
Robert Bernecky
conflicting types, likely caused by maxwlur 1 and/or maxspec 0
This unit test compiles okay with no command-line options, btw.
```
****** UTIndexSet::testisXIB( hidden, hidden, int, double, int, int, int): ...
Applying common subexpression elimination ...
Determine candida...
This unit test compiles okay with no command-line options, btw.
```
****** UTIndexSet::testisXIB( hidden, hidden, int, double, int, int, int): ...
Applying common subexpression elimination ...
Determine candidate functions for inlining ...
Inferring loop invariant variables ...
Applying type upgrade ...
./UTIndexSet.sac:227: abort:
loop variable "x" is being used inconsistently in function _dup_3625_sameIIB__Cond_0; conflicting types
are int[.] and #10092: in [ --, int[4]] le <> ge <>
Compilation failed while Running SAC optimizations.
sac2c -V
sac2c 1.3.3-MijasCosta-1157-g02676
build-type: DEBUG
built-by: "sac" at 2024-02-14T10:15:11
apex@medusa:/tmp/crud$ sac2c_d UTIndexSet.sac -maxspec 0 -maxwlur 1 -v4
[UTIndexSet.sac](/uploads/fcc2dd5aadc758b05b607c3bfe02dc42/UTIndexSet.sac)
```
Sven-Bodo Scholz
Sven-Bodo Scholz
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2383
Complexity problem in FUNDEF_WRAPPERTYPE
2024-02-15T10:17:47Z
Thomas Koopman
Complexity problem in FUNDEF_WRAPPERTYPE
The following program is OOM-killed with 40GB of available space in `elim_alpha_types.c`
```
use Array: all;
use StdIO: all;
use Math: all;
#define CATEGORIES 10
inline
float[d:shp] averageOuter(float[n,d:shp] array)
{
return {iv ->...
The following program is OOM-killed with 40GB of available space in `elim_alpha_types.c`
```
use Array: all;
use StdIO: all;
use Math: all;
#define CATEGORIES 10
inline
float[d:shp] averageOuter(float[n,d:shp] array)
{
return {iv -> sum({[i] -> array[[i]++iv] | [i] < [n]}) | iv < shp} / tof(n);
}
inline
int MaxPos( float[.,.,.,.,.] output)
{
n = shape( output)[0];
max = output[[0,0,0,0,0]];
res = 0;
for( i=0; i<n; i++)
if( output[[i,0,0,0,0]] > max) {
max = output[[i,0,0,0,0]];
res = i;
}
return res;
}
inline
float MeanSquaredError( float[*] result, float[*] labels)
{
return sum ( 0.5f * ( labels - result) * ( labels - result) );
}
#define sumOuter(xo, xi, e, shpi, shpo) \
{xi -> with { \
(0 * shpo <= xo < shpo): (e)[xi]; \
}: fold(+, 0f) \
| xi < shpi}
/* In `Fortran' notation x(i:i+n+1) where i, n are vectors and the range
* is inclusive, exclusive. */
inline
float[d:n1] slide(int[d] i, float[d:m] x, int[d] n) | all(n1 == n + 1)
| all(n + 1 + i <= m)
{
return {iv -> x[iv + i] | iv < n + 1};
}
inline
float[d:mn] backslide(int[d] i, float[d:n1] y, int[d] mn)
| all(i < 1 + mn - n1)
{
return {iv -> 0f | iv < i;
iv -> y[iv - i] | i <= iv < n1 + i;
iv -> 0f | n1 + i <= iv < mn};
}
inline
float[*] logistics(float[*] x)
{
return {iv -> 1f / (1f + exp(-x[iv]))};
}
inline
float[n:oshp,n:ishp] block(float[d:shp] x, int[n] ishp)
| all(shp % ishp == 0)
| all(oshp * ishp == shp)
| (2 * n == d)
{
return {iv -> tile(ishp, iv * ishp, x) | iv < shp / ishp};
}
inline
float[*] unblock(float[*] a, int[.] bshp)
{
shp = drop(-shape(bshp), shape(a)) * bshp;
return {iv -> a[(iv / bshp) ++ (iv % bshp)] | iv < shp};
}
inline
float[n:ishp] selb(float[d:shp] x, int[n] iv, int[n] ishp)
| (2 * n == d)
| all(iv < shp / ishp)
{
return block(x, ishp)[iv];
}
#define IMAPS(iv, e, shp) {iv -> (e)[[0]] | iv < shp}
float sels(float[d:shp] x, int[d] iv)
{
return x[iv];
}
noinline
float[6, 5, 5], float[6], float[12, 6, 5, 5], float[12],
float[CATEGORIES, 12, 1, 4, 4], float[CATEGORIES], float
TrainZhang(float[28, 28] inp, float[6, 5, 5] k1, float[6] b1,
float[12, 6, 5, 5] k2, float[12] b2,
float[CATEGORIES, 12, 1, 4, 4] fc, float[CATEGORIES] b,
float[CATEGORIES, 1, 1, 1, 1] target)
{
c11 = { x20 -> (sumOuter(x21, x22, (slide(x21, inp, [23, 23])) * (IMAPS(x23, (sels((k1)[x20], x21)), ([24, 24]))), ([5, 5]), ([24, 24]))) + (IMAPS(x24, (sels(b1, x20)), ([24, 24]))) | x20 < [6] };
c1 = logistics(c11);
s1 = { x16 -> IMAPS(x17, ((sumOuter(x18, x19, sels(selb((c1)[x16], x17, [2, 2]), x18), ([2, 2]), ([1]))) / 4), ([12, 12])) | x16 < [6] };
c21 = { x11 -> (sumOuter(x12, x13, (slide(x12, s1, [0, 7, 7])) * (IMAPS(x14, (sels((k2)[x11], x12)), ([1, 8, 8]))), ([6, 5, 5]), ([1, 8, 8]))) + (IMAPS(x15, (sels(b2, x11)), ([1, 8, 8]))) | x11 < [12] };
c2 = logistics(c21);
s2 = { x6 -> { x7 -> IMAPS(x8, ((sumOuter(x9, x10, sels(selb(((c2)[x6])[x7], x8, [2, 2]), x9), ([2, 2]), ([1]))) / 4), ([4, 4])) | x7 < [1] } | x6 < [12] };
r1 = { x1 -> (sumOuter(x2, x3, (slide(x2, s2, [0, 0, 0, 0])) * (IMAPS(x4, (sels((fc)[x1], x2)), ([1, 1, 1, 1]))), ([12, 1, 4, 4]), ([1, 1, 1, 1]))) + (IMAPS(x5, (sels(b, x1)), ([1, 1, 1, 1]))) | x1 < [10] };
r = logistics(r1);
ddr = 1f;
ddr1 = (ddr) * ((r1) * ((1f) + (-(r1))));
dds2 = sumOuter(x1, x2, sumOuter(x3, x4, backslide(x3, IMAPS(x5, ((sels((ddr1)[x1], x5)) * (sels((fc)[x1], x3))), ([1, 1, 1, 1])), [12, 1, 4, 4]), ([12, 1, 4, 4]), ([12, 1, 4, 4])), ([10]), ([12, 1, 4, 4]));
ddc2 = { x1 -> { x2 -> unblock({ x3 -> IMAPS(x4, ((sels(((dds2)[x1])[x2], x3)) / 4), ([2, 2])) | x3 < [4, 4] }, [2, 2]) | x2 < [1] } | x1 < [12] };
ddc21 = (ddc2) * ((c21) * ((1f) + (-(c21))));
dds1 = sumOuter(x1, x2, sumOuter(x3, x4, backslide(x3, IMAPS(x5, ((sels((ddc21)[x1], x5)) * (sels((k2)[x1], x3))), ([1, 8, 8])), [6, 12, 12]), ([6, 5, 5]), ([6, 12, 12])), ([12]), ([6, 12, 12]));
ddc1 = { x1 -> unblock({ x2 -> IMAPS(x3, ((sels((dds1)[x1], x2)) / 4), ([2, 2])) | x2 < [12, 12] }, [2, 2]) | x1 < [6] };
ddc11 = (ddc1) * ((c11) * ((1f) + (-(c11))));
ddb = IMAPS(x1, (sumOuter(x2, x3, sels((ddr1)[x1], x2), ([1, 1, 1, 1]), ([1]))), ([10]));
ddfc = { x1 -> IMAPS(x2, (sumOuter(x3, x4, (sels((ddr1)[x1], x3)) * (sels(slide(x2, s2, [0, 0, 0, 0]), x3)), ([1, 1, 1, 1]), ([1]))), ([12, 1, 4, 4])) | x1 < [10] };
ddb2 = IMAPS(x1, (sumOuter(x2, x3, sels((ddc21)[x1], x2), ([1, 8, 8]), ([1]))), ([12]));
ddk2 = { x1 -> IMAPS(x2, (sumOuter(x3, x4, (sels((ddc21)[x1], x3)) * (sels(slide(x2, s1, [0, 7, 7]), x3)), ([1, 8, 8]), ([1]))), ([6, 5, 5])) | x1 < [12] };
ddb1 = IMAPS(x1, (sumOuter(x2, x3, sels((ddc11)[x1], x2), ([24, 24]), ([1]))), ([6]));
ddk1 = { x1 -> IMAPS(x2, (sumOuter(x3, x4, (sels((ddc11)[x1], x3)) * (sels(slide(x2, inp, [23, 23]), x3)), ([24, 24]), ([1]))), ([5, 5])) | x1 < [6] };
ddinp = sumOuter(x1, x2, sumOuter(x3, x4, backslide(x3, IMAPS(x5, ((sels((ddc11)[x1], x5)) * (sels((k1)[x1], x3))), ([24, 24])), [28, 28]), ([5, 5]), ([28, 28])), ([6]), ([28, 28]));
error = MeanSquaredError(r, target);
return (ddk1, ddb1, ddk2, ddb2, ddfc, ddb, error);
}
int main()
{
k1 = genarray([6, 5, 5], 1f / 25f);
b1 = genarray([6], 1f / 6f);
k2 = genarray([12, 6, 5, 5], 1f / 150f);
b2 = genarray([12], 1f / 12f);
fc = genarray([CATEGORIES, 12, 1, 4, 4], 1f / 192f);
b = genarray([CATEGORIES], 1f / tof(CATEGORIES));
target = genarray([CATEGORIES,1,1,1,1], 0f);
inp = reshape([28, 28], tof(iota(28 * 28)));
d_k1, d_b1, d_k2, d_b2, d_fc, d_b, err = TrainZhang(inp, k1, b1, k2, b2, fc, b, target);
print(d_k1);
return 0;
}
```
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2382
Constant folding does not optimise sel(jv, {iv -> expr(iv) | iv < ub})
2024-02-15T10:19:35Z
Thomas Koopman
Constant folding does not optimise sel(jv, {iv -> expr(iv) | iv < ub})
Consider the following program.
```
inline
int[d:n1] slide(int[d] i, int[d:m] x, int[d] n1) | all(n1 + i <= m)
{
return {iv -> _sel_VxA_(_add_VxV_(iv, i), x)
| iv < n1};
}
int +(int a, int b)
{
return _add_SxS_(a, b)...
Consider the following program.
```
inline
int[d:n1] slide(int[d] i, int[d:m] x, int[d] n1) | all(n1 + i <= m)
{
return {iv -> _sel_VxA_(_add_VxV_(iv, i), x)
| iv < n1};
}
int +(int a, int b)
{
return _add_SxS_(a, b);
}
noinline
int[*] id(int[*] x)
{
return x;
}
int main()
{
inp = id(with {}: genarray([28, 28], 2));
#if 1
bla = {x22 -> with {
([0, 0] <= x21 < [5, 5]): _sel_VxA_(x22, slide(x21, inp, [23, 23]));
}: fold(+, 0)
| x22 < [24, 24]};
#else
bla = {x22 -> with {
([0, 0] <= x21 < [5, 5]): _sel_VxA_(_add_VxV_(x22, x21), inp);
}: fold(+, 0)
| x22 < [24, 24]};
#endif
res = _sel_VxA_([0, 0], bla);
return res;
}
```
Compiling the following example with `sac2c_d -bopt -printfun main` gives
```
/****************************************************************************
* _MAIN::main(...) [ body ]
****************************************************************************/
int _MAIN::main()
/*
* main :: ---
*/
{
int _ivesli_2836 { , NN } ;
int _ivesli_2835 { , NN } ;
int _ivesli_2834 { , NN } ;
int _ivesli_2832 { , NN } ;
int _wlidx_2816__flat_91 { , NN } ;
int _wlidx_2815_bla { , NN } ;
int _wlidx_2814__flat_33 { , NN } ;
int _pinl_405__eat_107 { , NN } ;
int _pinl_404__eat_106 { , NN } ;
int _pinl_403__mose_8__SSA0_1 { , NN } ;
int[2] _pinl_402_iv { , NN } ;
int _ea_327__flat_90 { , NN } ;
int _ea_326__mose_9__SSA0_1 { , NN } ;
int _eat_105 { , NN } ;
int _eat_104 { , NN } ;
int _eat_103 { , NN } ;
int _eat_102 { , NN } ;
int _eat_99 { , NN } ;
int _eat_98 { , NN } ;
int _mose_9__SSA0_1 { , NN } ;
int[2] x21__SSA0_1 { , NN } ;
int res { , NN } ;
int[24,24] bla { , NN } ;
int[2] x22 { , NN } ;
int[28,28] inp { , NN } ;
int[2] _hzgwl_12 { , NN } ;
int[23,23] _flat_91 { , NN } ;
int _flat_90 { , NN } ;
int{0} _flat_39 { , NN } ;
int{2} _flat_37 { , NN } ;
int[28,28] _flat_33 { , NN } ;
_flat_39 = 0;
_flat_37 = 2;
_flat_33 = with /** FOLDABLE (all gen's const) **/
/** REFERENCED: 1 (total num refs) **/
{
/* Partn */
([ 0, 0 ] <= _hzgwl_12=[_eat_99, _eat_98] (IDXS:_wlidx_2814__flat_33) < [ 28, 28 ] genwidth [ 28, 28 ])
{
} : _flat_37 ;
} :
genarray( [ 28, 28 ], _flat_37, IDX(_wlidx_2814__flat_33));
inp = _MAIN::id( _flat_33) ;
bla = with /** FOLDABLE (all gen's const) **/
/** REFERENCED: 1 (total num refs) **/
{
/* Partn */
([ 0, 0 ] <= x22=[_eat_103, _eat_102] (IDXS:_wlidx_2815_bla) < [ 24, 24 ] genwidth [ 24, 24 ])
{
_ivesli_2836 = _idxs2offset_( [ 23, 23 ], _eat_103, _eat_102);
_mose_9__SSA0_1 = with /** FOLDABLE (all gen's const) **/
/** REFERENCED: 1 (total num refs) **/
{
/* Partn */
([ 0, 0 ] <= x21__SSA0_1=[_eat_105, _eat_104] < [ 5, 5 ] genwidth [ 5, 5 ])
{
_ea_326__mose_9__SSA0_1 = _accu_( x21__SSA0_1, _flat_39);
_ivesli_2832 = _idxs2offset_( [ 28, 28 ], _eat_105, _eat_104);
_flat_91 = with /** FOLDABLE (all gen's const) **/
/** REFERENCED: 1 (total num refs) **/
{
/* Partn */
([ 0, 0 ] <= _pinl_402_iv=[_pinl_405__eat_107, _pinl_404__eat_106] (IDXS:_wlidx_2816__flat_91) < [ 23, 23 ] genwidth [ 23, 23 ])
{
_ivesli_2834 = _idxs2offset_( [ 28, 28 ], _pinl_405__eat_107, _pinl_404__eat_106);
_ivesli_2835 = _add_SxS_( _ivesli_2832, _ivesli_2834);
_pinl_403__mose_8__SSA0_1 = _idx_sel_( _ivesli_2835, inp);
} : _pinl_403__mose_8__SSA0_1 ;
} :
genarray( [ 23, 23 ], _flat_39, IDX(_wlidx_2816__flat_91));
_flat_90 = _idx_sel_( _ivesli_2836, _flat_91);
_ea_327__flat_90 = _add_SxS_( _ea_326__mose_9__SSA0_1, _flat_90);
} : _ea_327__flat_90 ;
} :
fold( _MAIN::+(), _flat_39);
} : _mose_9__SSA0_1 ;
} :
genarray( [ 24, 24 ], _flat_39, IDX(_wlidx_2815_bla));
res = _idx_sel_( _flat_39, bla);
return( res);
}
/*-----------------------------------------------*/
```
so `slide` is computed every iteration of the fold-loop, whereas I would have expected a simple selection `inp[x21 + x22]`, as in the commented out version.
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2381
Assertion "(actual_cls != C_scl) && (formal_cls != C_scl)" fails
2024-03-03T10:26:09Z
Thomas Koopman
Assertion "(actual_cls != C_scl) && (formal_cls != C_scl)" fails
Not so minimal example
```
use MathArray: all;
use Structures: all;
use StdIO: all;
#define CATEGORIES 10
#define TRAIN_SIZE 60000
inline
float[n:shpi] rsum(int m, float[m:shpo,n:shpi] x)
{
return {iv -> sum({jv -> x[jv++iv] | jv < ...
Not so minimal example
```
use MathArray: all;
use Structures: all;
use StdIO: all;
#define CATEGORIES 10
#define TRAIN_SIZE 60000
inline
float[n:shpi] rsum(int m, float[m:shpo,n:shpi] x)
{
return {iv -> sum({jv -> x[jv++iv] | jv < shpo}) | iv < shpi};
}
noinline
float[m:mshp,n:oshp,b:bshp] MultiConv(float[n:ishp,b:bshp] in,
float[m:mshp,n:wshp] weights,
float[m:mshp] bias)
| all(oshp == ishp - wshp + 1)
{
x = {iv -> Convolve(in, weights[iv]) + bias[iv] | iv < mshp};
print(shape(x));
return x;
}
noinline
float[n:oshp,b:bshp] Convolve(float[n:ishp,b:bshp] in, float[n:wshp] weights)
{
oshp = ishp - wshp + 1;
return {iv -> rsum(n, {jv -> weights[jv] * in[iv + jv]}) | iv < oshp};
}
inline
float average( float[*] array)
{
return sum( array) / tof( prod( shape( array)));
}
inline
float[*] AveragePool( float[*] in, int[.] filter)
//
// assert( dim(in) >= shape(filter)[0] )
// assert( shape(out) == shape(in)/filter )
//
{
ones = genarray( [dim( in)], 1);
filter = drop( shape( filter), ones) ++ filter;
shp = shape( in) / filter;
/*
* out = { iv -> average( { ov -> in[iv+ov] | ov < filter})
* | iv < shp};
*/
out = with {
(. <= iv <= .) : average( with {
(. <= ov <= .) : in[iv*filter+ov];
} : genarray( filter, 0f));
} : genarray( shp, 0f);
return out;
}
inline
float[*] BackAveragePool( float[*] d_out, int[.] filter )
{
ones = genarray( [dim( d_out)], 1);
filter = drop( shape( filter), ones) ++ filter;
shp = shape( d_out) * filter;
d_in = with {
(. <= iv <=.) : d_out[iv/filter] / tof( prod( filter));
} : genarray( shp, 0f);
return d_in;
}
inline
float[*] BackWeights2( float[*] d_out, float[*] weights, float[*] in)
{
return with {
( . <= ov <= .) :
with {
(0*shape( d_out) <= iv < shape( d_out)) : in[ iv+ov] * d_out[iv];
} : fold( +, 0f );
} : genarray( shape( weights), 0f);
}
float[*], float[*], float[*]
BackMultiConv( float[*] d_out, float[*] weights, float[*] in, float[*] bias)
{
shp_act_map = take( -[dim(in)], shape(weights));
shp_maps = drop( -[dim(in)], shape(weights));
d_in = with {
( . <= iv <= .) :
with {
(0*shp_maps <= ov < shp_maps) {
lb = max( 0*shp_act_map, iv - take( -[dim(in)], shape(d_out)) + 1);
ub = min( shp_act_map, iv+1 );
} : with {
( lb <= ov2 < ub) : weights[ov ++ ov2] * d_out[ov ++ (iv-ov2)];
} : fold( +, 0f);
} : fold( +, 0f);
} : genarray( shape(in), 0f);
d_weights = with {
(. <= iv <= .) : BackWeights2( d_out[iv], weights[iv], in);
} : genarray( shp_maps, genarray( take( -[dim(in)], shape(weights)), 0f));
d_bias = with {
(. <= iv <= .) : sum( d_out[iv]);
} : genarray( shp_maps, 0f);
return ( d_in, d_weights, d_bias);
}
inline
float[*] BackLogistic( float[*] d_out, float[*] out)
{
return d_out * out * (1f - out);
}
inline
float[6,5,5], float[6], float[12,6,5,5], float[12], float[CATEGORIES,12,1,4,4], float[CATEGORIES], float
TrainZhang( float[28,28] in, float[6,5,5] k1, float[6] b1,
float[12,6,5,5] k2, float[12] b2,
float[CATEGORIES,12,1,4,4] fc, float[CATEGORIES] b,
float[CATEGORIES,1,1,1,1] target)
{
float[6,24,24] c1, d_c1;
float[6,12,12] s1, d_s1;
float[12,1,8,8] c2, d_c2;
float[12,1,4,4] s2, d_s2;
float[CATEGORIES,1,1,1,1] out, d_out;
c1 = MultiConv( in, k1, b1 );
s1 = AveragePool( c1, [2,2]);
c2 = MultiConv( s1, k2, b2);
s2 = AveragePool( c2, [2,2]);
out = MultiConv( s2, fc, b);
d_out = out - target;
error = 0f;
d_s2, d_fc, d_b = BackMultiConv( BackLogistic( d_out, out), fc, s2, b);
d_c2 = BackAveragePool( d_s2, [2,2]);
d_s1, d_k2, d_b2 = BackMultiConv( BackLogistic( d_c2, c2), k2, s1, b2);
d_c1 = BackAveragePool( d_s1, [2,2]);
_, d_k1, d_b1 = BackMultiConv( BackLogistic( d_c1, c1), k1, in, b1);
return ( d_k1, d_b1, d_k2, d_b2, d_fc, d_b, error);
}
int main()
{
batchsize = 100;
k1 = genarray( [6,5,5], 1f/25f);
b1 = genarray( [6], 1f/6f);
k2 = genarray( [12,6,5,5], 1f/150f);
b2 = genarray( [12], 1f/12f);
fc = genarray( [CATEGORIES,12,1,4,4], 1f/192f);
b = genarray( [CATEGORIES], 1f/tof(CATEGORIES));
training_images = genarray([TRAIN_SIZE,28,28], 0f);
training_labels = genarray([TRAIN_SIZE], 3);
error = 0d;
delta_k1, delta_b1, delta_k2, delta_b2, delta_fc, delta_b, berr =
with {
([0] <= iv < [batchsize]) {
in = training_images[iv];
target = genarray([CATEGORIES,1,1,1,1], 0f);
target[[training_labels[iv],0,0,0,0]] = 1f;
d_k1, d_b1, d_k2, d_b2, d_fc, d_b, err =
TrainZhang(in, k1, b1, k2, b2, fc, b, target);
}: (d_k1, d_b1, d_k2, d_b2, d_fc, d_b, err);
}: (fold(+, 0f * k1),
fold(+, 0f * b1),
fold(+, 0f * k2),
fold(+, 0f * b2),
fold(+, 0f * fc),
fold(+, 0f * b),
fold(+, 0f));
k1 = k1 - delta_k1;
b1 = b1 - delta_b1;
k2 = k2 - delta_k2;
b2 = b2 - delta_b2;
fc = fc - delta_fc;
b = b - delta_b;
error += tod(berr);
print(k1);
return 0;
}
```
using
```
sac2c 1.3.3-MijasCosta-1153-g14eb
build-type: DEBUG
built-by: "thomas" at 2024-02-12T12:49:22
```
with `check -c`
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2380
check tc false positive
2024-02-14T10:10:58Z
Thomas Koopman
check tc false positive
The following program gives `Type pattern error in application of averageOuter: feature `d:shp' in return value does not match feature `shp' in arrayCompilation failed while Running SAC optimizations`. I think the program is correct howe...
The following program gives `Type pattern error in application of averageOuter: feature `d:shp' in return value does not match feature `shp' in arrayCompilation failed while Running SAC optimizations`. I think the program is correct however. It should average `batchsize` many copies of `k1`.
```
use Array: all;
inline
float[d:shp] averageOuter(float[n,d:shp] array)
{
return {iv -> sum({[i] -> array[[i]++iv] | [i] < [n]}) | iv < shp};
}
int main()
{
batchsize = 100;
k1 = with {}: genarray([6, 5, 5], 1f / 25f);
/* [batchsize, 6, 5, 5] */
bd_k1 = {iv -> k1 | iv < [batchsize]};
k1 = averageOuter(bd_k1);
return _toi_S_(_sel_VxA_([0, 0, 0], k1));
}
```
Compiled with arguments `-check tc`
```
sac2c 1.3.3-MijasCosta-1153-g14eb
build-type: DEBUG
built-by: "thomas" at 2024-02-12T12:49:22
```
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2379
sac2c crash with -doawlf
2024-02-12T18:25:55Z
Robert Bernecky
sac2c crash with -doawlf
Title says it all.
```
cat crud2.sac
use Array:{iota};
noinline
int[*] myReshape (int[.] shp, int[*] data)
{
return _reshape_VxA_(shp, data);
}
int main()
{ z = myReshape( [5], iota(5));
StdIO::show(z);
return(0);
}
apex@medu...
Title says it all.
```
cat crud2.sac
use Array:{iota};
noinline
int[*] myReshape (int[.] shp, int[*] data)
{
return _reshape_VxA_(shp, data);
}
int main()
{ z = myReshape( [5], iota(5));
StdIO::show(z);
return(0);
}
apex@medusa:/tmp$ sac2c crud2.sac -doawlf
Warning:
AWLF is enabled: -ecc enabled.
Warning:
AWLF is enabled: -extrema enabled.
Warning:
AWLF is enabled: -maxoptcyc=20
Internal compiler error
Assertion "res == NULL || NODE_TYPE (res) == N_array" failed at /home/sac/sac2c/src/libsac2c/arrayopt/with_loop_utilities.c:289 -- Array conversion failed!
Please file a bug at: https://gitlab.sac-home.org/sac-group/sac2c/-/issues
apex@medusa:/tmp$ sac2c -V
sac2c 1.3.3-MijasCosta-1149-g4237
build-type: RELEASE
built-by: "sac" at 2024-02-09T11:28:36
```
Sven-Bodo Scholz
Sven-Bodo Scholz
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2378
check c problem ICM undefined!
2024-02-10T17:06:10Z
Sven-Bodo Scholz
check c problem ICM undefined!
When compiling
```
noinline
int[*] myReshape (int[.] shp, int[*] data)
{
return _reshape_VxA_(shp, data);
}
int main() {
x = myReshape ([1], 42);
return _sel_VxA_([0], x);
}
```
with `sac2c -check c`
we get
```
** 22: Cr...
When compiling
```
noinline
int[*] myReshape (int[.] shp, int[*] data)
{
return _reshape_VxA_(shp, data);
}
int main() {
x = myReshape ([1], 42);
return _sel_VxA_([0], x);
}
```
with `sac2c -check c`
we get
```
** 22: Creating binary code ...
/Volumes/Users/sbs/Dropbox/BodoSac/ruben/a.out.c:1476:265: error: use of undeclared identifier '_ICM_IS_UNDEFINED_'
SAC_RuntimeWarningLoc ("./bug1.sac", 4, 12, "Arrays `" "SACp_emal_6042__idc_1216""' with shape %s ""And `" "SACl_data""' with shape %s ""do not adhere to prod matches prod shape constraint", SAC_PrintShape (SACp_emal_6042__idc_1216__desc), SAC_PrintShape (_ICM_IS_UNDEFINED_));
^
1 error generated.
```
One can also use `-noprelude` alongside....
Without `-check c` it compiles and runs fine.
When inlining it also runs fine.
Jordy Aaldering
Jordy Aaldering
https://gitlab.sac-home.org/sac-group/sac2c/-/issues/2377
Complexity problem in IVD
2024-02-11T10:13:20Z
Thomas Koopman
Complexity problem in IVD
IVD checks for each `N_spids` whether a variable declaration exists, inserting one if necessary. This search is a linear traversal of the list of variable declarations. If we have `n` variables without declaration, that means we check `s...
IVD checks for each `N_spids` whether a variable declaration exists, inserting one if necessary. This search is a linear traversal of the list of variable declarations. If we have `n` variables without declaration, that means we check `sum_{i = 1}^n i = n(n + 1)/2 \in O(n^2)` times. For the program below, the phase FLAT generates about 100K variables without declaration, leading to this phase taking about 20 minutes.
```
use Array: all;
use Math: all;
use StdIO: all;
#define CATEGORIES 10
inline
float[n:shpi] sumOuter(int m, float[m:shpo,n:shpi] x)
{
return {iv -> sum({jv -> x[jv++iv] | jv < shpo}) | iv < shpi};
}
inline
float[d:n1] slide(int[d] i, float[d:m] x, int[d] n) | all(n1 == n + 1)
| all(n + 1 + i <= m)
{
return {iv -> x[iv + i] | iv < n + 1};
}
inline
float[d:mn] backslide(int[d] i, float[d:n1] y, int[d] mn)
| all(i < 1 + mn - n1)
{
return {iv -> 0f | iv < i;
iv -> y[iv - i] | i <= iv < n1 + i;
iv -> 0f | n1 + i <= iv < mn};
}
inline
float[*] logistics(float[*] x)
{
return {iv -> 1f / (1f + exp(-x[iv]))};
}
inline
float[n:oshp,n:ishp] block(float[d:shp] x, int[n] ishp)
| all(shp % ishp == 0)
| all(oshp * ishp == shp)
| (2 * n == d)
{
return {iv -> tile(ishp, iv * ishp, x) | iv < shp / ishp};
}
/* TODO: type patterns */
inline
float[*] unblock(float[*] a, int[.] bshp)
{
shp = drop(-shape(bshp), shape(a)) * bshp;
return {iv -> a[(iv / bshp) ++ (iv % bshp)] | iv < shp};
}
inline
float[n:ishp] selb(float[d:shp] x, int[n] iv, int[n] ishp)
| (2 * n == d)
| all(iv < shp / ishp)
{
return block(x, ishp)[iv];
}
int main()
{
/* Variables */
inp = {[i, j] -> tof(i * j) | [i, j] < [28, 28]};
k1 = genarray( [6,5,5], 1f/25f);
b1 = genarray( [6], 1f/6f);
k2 = genarray( [12,6,5,5], 1f/150f);
b2 = genarray( [12], 1f/12f);
fc = genarray( [CATEGORIES,12,1,4,4], 1f/192f);
b = genarray( [CATEGORIES], 1f/tof(CATEGORIES));
/* Generated code with minor adjustments:
UTF8 -> ASCII
4 -> 4f (as SaC does not coerce types)
one -> 1f
backlide -> backslide
*/
/* Shape [6, 24, 24] */
c11 = { x16 -> (sumOuter(2, { x17 -> (slide(x17, inp, [23, 23])) * ({ x18 -> ((k1)[x16])[x17] | x18 < [24, 24] }) | x17 < [5, 5]})) + ({ x19 -> (b1)[x16] | x19 < [24, 24] }) | x16 < [6] };
printf("c11\n");
print(c11);
/* Shape [6, 24, 24] */
c1 = logistics(c11);
printf("c1\n");
print(c1);
/* Shape [6, 12, 12] */
s1 = { x13 -> { x14 -> (sumOuter(2, { x15 -> (selb((c1)[x13], x14, [2, 2]))[x15] | x15 < [2, 2]})) / 4f | x14 < [12, 12] } | x13 < [6] };
printf("s1\n");
print(s1);
/* Shape [12, 1, 8, 8] */
c21 = { x9 -> (sumOuter(3, { x10 -> (slide(x10, s1, [0, 7, 7])) * ({ x11 -> ((k2)[x9])[x10] | x11 < [1, 8, 8] }) | x10 < [6, 5, 5]})) + ({ x12 -> (b2)[x9] | x12 < [1, 8, 8] }) | x9 < [12] };
printf("c21\n");
print(c21);
/* Shape [12, 1, 8, 8] */
c2 = logistics(c21);
printf("c2\n");
print(c2);
/* Shape [12, 1, 4, 4] */
s2 = { x5 -> { x6 -> { x7 -> (sumOuter(2, { x8 -> (selb(((c2)[x5])[x6], x7, [2, 2]))[x8] | x8 < [2, 2]})) / 4f | x7 < [4, 4] } | x6 < [1] } | x5 < [12] };
printf("s2\n");
print(s2);
/* Shape [10, 1, 1, 1, 1] */
r1 = { x1 -> (sumOuter(4, { x2 -> (slide(x2, s2, [0, 0, 0, 0])) * ({ x3 -> ((fc)[x1])[x2] | x3 < [1, 1, 1, 1] }) | x2 < [12, 1, 4, 4]})) + ({ x4 -> (b)[x1] | x4 < [1, 1, 1, 1] }) | x1 < [10] };
printf("r1\n");
print(r1);
/* Shape [10, 1, 1, 1, 1] */
r = logistics(r1);
printf("r\n");
print(r);
ddr = 1f;
/* Shape [10, 1, 1, 1, 1] */
ddr1 = ddr * (r1 * (1f + -r1));
printf("ddr1\n");
print(ddr1);
/* Shape [12, 1, 4, 4] */
dds2 = sumOuter(1, { x1 -> sumOuter(4, { x2 -> backslide(x2, { x3 -> (((ddr1)[x1])[x3]) * (((fc)[x1])[x2]) | x3 < [1, 1, 1, 1] }, [12, 1, 4, 4]) | x2 < [12, 1, 4, 4]}) | x1 < [10]});
printf("dds2\n");
print(dds2);
/* Shape [12, 1, 8, 8] */
ddc2 = { x1 -> { x2 -> unblock({ x3 -> { x4 -> ((((dds2)[x1])[x2])[x3]) / 4f | x4 < [2, 2] } | x3 < [4, 4] }, [2, 2]) | x2 < [1] } | x1 < [12] };
printf("ddc2\n");
print(ddc2);
/* Shape [12, 1, 8, 8] */
ddc21 = ddc2 * (c21 * (1f + -c21));
printf("ddc21\n");
print(ddc21);
/* Shape [6, 12, 12] */
dds1 = sumOuter(1, { x1 -> sumOuter(3, { x2 -> backslide(x2, { x3 -> (((ddc21)[x1])[x3]) * (((k2)[x1])[x2]) | x3 < [1, 8, 8] }, [6, 12, 12]) | x2 < [6, 5, 5]}) | x1 < [12]});
printf("dds1\n");
print(dds1);
/* Shape [6, 24, 24] */
ddc1 = { x1 -> unblock({ x2 -> { x3 -> (((dds1)[x1])[x2]) / 4f | x3 < [2, 2] } | x2 < [12, 12] }, [2, 2]) | x1 < [6] };
printf("ddc1\n");
print(ddc1);
/* Shape [6, 24, 24] */
ddc11 = (ddc1) * (c11 * (1f + -c11));
printf("ddc11\n");
print(ddc11);
ddb = { x1 -> sumOuter(4, { x2 -> ((ddr1)[x1])[x2] | x2 < [1, 1, 1, 1]}) | x1 < [10] };
printf("ddb\n");
print(ddb);
ddfc = { x1 -> { x2 -> sumOuter(4, { x3 -> (((ddr1)[x1])[x3]) * ((slide(x2, s2, [0, 0, 0, 0]))[x3]) | x3 < [1, 1, 1, 1]}) | x2 < [12, 1, 4, 4] } | x1 < [10] };
printf("ddfc\n");
print(ddfc);
ddb2 = { x1 -> sumOuter(3, { x2 -> ((ddc21)[x1])[x2] | x2 < [1, 8, 8]}) | x1 < [12] };
printf("ddb2\n");
print(ddb2);
ddk2 = { x1 -> { x2 -> sumOuter(3, { x3 -> (((ddc21)[x1])[x3]) * ((slide(x2, s1, [0, 7, 7]))[x3]) | x3 < [1, 8, 8]}) | x2 < [6, 5, 5] } | x1 < [12] };
printf("ddk2\n");
print(ddk2);
ddb1 = { x1 -> sumOuter(2, { x2 -> ((ddc11)[x1])[x2] | x2 < [24, 24]}) | x1 < [6] };
printf("ddb1\n");
print(ddb1);
ddk1 = { x1 -> { x2 -> sumOuter(2, { x3 -> (((ddc11)[x1])[x3]) * ((slide(x2, inp, [23, 23]))[x3]) | x3 < [24, 24]}) | x2 < [5, 5] } | x1 < [6] };
printf("ddk1\n");
print(ddk1);
ddinp = sumOuter(1, { x1 -> sumOuter(2, { x2 -> backslide(x2, { x3 -> (((ddc11)[x1])[x3]) * (((k1)[x1])[x2]) | x3 < [24, 24] }, [28, 28]) | x2 < [5, 5]}) | x1 < [6]});
printf("ddinp\n");
print(ddinp);
return 0;
}
```