I'd like to propose a different kind of golfing challenge to this community:

In the language and framework of your choice, design and train a neural network that, given $(x_1, x_2)$ calculates their product $x_1 cdot x_2$ for all integers $x_1, x_2$ between (and including) $-10$ and $10$.

Performance Goal

To qualify, your model may not deviate by more than $0.5$ from the correct result on any of those entries.

Rules

Your model

• must be a 'traditional' neural network (a node's value is calculated as a weighted linear combination of some of the nodes in a previous layer followed by an activation function),


• may only use the following standard activation functions:
  
  
  1. 
     $mathrmlinear(x) = x$,
  
  
  2. 
     $mathrmsoftmax(vecx)_i = frace^x_isum_j e^x_j$,
  
  
  3. 
     $mathrmselu_alpha, beta(x) = begincases
     beta cdot x & text, if x > 0 \
     alpha cdot beta (e^x -1 ) & text, otherwise
     endcases$,
  
  
  4. 
     $mathrmsoftplus(x) = ln(e^x+1)$,
  
  
  5. 
     $mathrmleaky-relu_alpha(x) = begincases
     x & text, if x < 0 \
     alpha cdot x & text, otherwise
     endcases$,
  
  
  6. 
     $tanh(x)$,
  
  
  7. 
     $mathrmsigmoid(x) = frace^xe^x+1$,
  
  
  8. 
     $mathrmhard-sigmoid(x) = begincases
     0 & text, if x < -2.5 \
     1 & text, if x > -2.5 \
     0.2 cdot x + 0.5 & text, otherwise
     endcases$,
  
  
  9. $e^x$
  
  


• must take $(x_1, x_2)$ either as a tupel/vector/list/... of integers or floats as its only input,


• return the answer as an integer, float (or a suitable container, e.g. a vector or list, that contains this answer).

Your answer must include (or link to) all code necessary to check your results -- including the trained weights of your model.

Scoring

The neural network with the smallest number of weights (including bias weights) wins.

Enjoy!

33 31 weights

# Activation functions
sub hard $_[0] < -2.5 ? 0 : $_[0] > 2.5 ? 1 : 0.2 * $_[0] + 0.5 
sub linear $_[0] 

# Layer 0
sub inputA() $a 
sub inputB() $b 

# Layer 1
sub a15() hard(5*inputA) 

# Layer 2
sub a8() hard(-5*inputA + 75*a15 - 37.5) 

# Layer 3
sub aa() linear(-5*inputA + 75*a15 - 40*a8) 

# Layer 4
sub a4() hard(aa - 17.5) 

# Layer 5
sub a2() hard(aa - 20*a4 - 7.5) 

# Layer 6
sub a1() linear(0.2*aa - 4*a4 - 2*a2) 

# Layer 7
sub b15() hard(0.25*inputB - 5*a15) 
sub b8() hard(0.25*inputB - 5*a8) 
sub b4() hard(0.25*inputB - 5*a4) 
sub b2() hard(0.25*inputB - 5*a2) 
sub b1() hard(0.25*inputB - 5*a1) 

# Layer 8
sub output() linear(-300*b15 + 160*b8 + 80*b4 + 40*b2 + 20*b1 - 10*inputA) 

# Test
for $a (-10..10) 
 for $b (-10..10) 
 die if abs($a * $b - output) >= 0.5;
 


print "All OK";

Try it online!

This does long multiplication in (sorta) binary, and thus returns the exact result. It should be possible to take advantage of the 0.5 error window to golf this some more, but I'm not sure how.

Layers 1 to 6 decompose the first input in 5 "bits". For golfing reasons, we don't use actual binary. The most significant "bit" has weight -15 instead of 16, and when the input is 0, all the "bits" are 0.5 (which still works out fine, since it preserves the identity inputA = -15*a15 + 8*a8 + 4*a4 + 2*a2 + 1*a1).

21 13 weights

This is based on the polarization identity of bilinear forms which in the one dimensional real case reduces to the polynomial identity:

$$ xcdot y = frac(x+y)^2 - (x-y)^24$$

So y1 just computes [x+y, x-y] using a linear transformation, and y3 is just the absolute value of y1 as a preprocessing step for the next one: Then the "hard" part is computing the squares which I will explain below, and after that just computing a difference and scaling which is again a linear operation.

To compute the squares I use an exponential series $s$ which should be accurate for all integers $0,1,2,ldots,20$ within around $0.5$. This series is of the form

$$ textapprox_square(x) = sum_i=0^2 w_i exp(0.0001 cdot i cdot x)$$

where I just optimized for the weights W2 ($=(w_i)_i$). This whole approximation comprises again just two linear transformations with an exponential activation sandwiched in between. This approach results in a maximal deviation of about 0.02.

function p = net(x)
relu = @(x)max(0,x);

% Linear: 4 weights
y1 = [1,1;1,-1]*x;

% Linear + ReLu + Sum: 1 weight (the -1):
y2 = -1 * y1; 
y3 = relu(y1) + relu(y2);

% Linear + exp: 3 weights:
W1 = (0:2)*0.0001;
y4 = exp(y3 * W1); 

% Linear: 3 weights:
W2 = [99700114.4299316;-199400468.100687;99700353.6313757];
y5 = y4 * W2;

% Linear : 2 weights:
p = [0.25,-0.25]*y5;

end

Try it online!