For creating rectangular prism, use the same step as creating cube. Only change the measurement. For the base shape, create rectangle instead of square.
To create cylinder, you should know the base shape of cylinder, which is circle. So, click create --> center diameter circle. Input the diameter of the circle.