When operating in NPS compatibility mode, a routine written in NZPLSQL has specific limitations. The routine must return either an integer or a null, ensuring that the return type is strictly defined. Additionally, the routine cannot contain argument lists or variable arguments (varargs), limiting the flexibility in defining input parameters. These constraints are essential to maintain proper function and compatibility with the system's requirements.