Look up `binomial coefficient' on (for example) Wikipedia.
Using google for `binomial coefficient' gives you many
links like the one for Wikipedia.
MATLAB uses the function nchoosek for binomial coefficients.
So nchoosek(5,2) returns 10.
For small k, like a fixed k satisfying 0 <= k <= 3, you can
write nchoosek(n,k) as a simple polynomial in n.

Consider abstract matrix
A = [a 1 0 0]
    [0 a 1 0]
    [0 0 a 1]
    [0 0 0 a]

1. Compute A^1 = A, A^2 = A*A, A^3 = A*A*A, and A^4 =
A*A*A*A.

2. Give the correct formula for A^n, where n is an
arbitrary positive integer.

In MATLAB, the null space N(B) of a matrix B is the span of null(B),
and the rangle R(B) of a matrix B is is the span of colspace(B).

3. and 4. Compute the null space N(B) of abstract matrix B and
the range R(B) of abstract matrix B, where
B = [0 1 0 0]
    [0 a 1 0]
    [0 0 a 1]
    [0 0 0 0]
    [0 0 0 b]

For this final homework, you have to look up with MATLAB help how to compute
eigenvalues and eigenvectors. Experiment a bit first, for example with easy
tiny matrices.

5. Find eigenvalues and eigenvectors of abstract matrix
C = [      U        0       0            2-2*U   ]
    [(2*M)/5-2/5    M   (2*M)/5-2/5   3/5-(3*M)/5]
    [   1-U         0       1            2*U-2   ]
    [    0          0       0               i    ]
Write your resulting diagonal matrix D as a product S^(-1) * C * S.