You can only set ustom size, if the printer support this.
To do this:
- in the report designer, go to File > Report Options and select printer
- in File > Page Options set the size: User (U)
- below the combobox enter width and height of the page in mm