You can pick up an M4 with a RIS already attached, and you can swap the RIS with other handguards, but you cannot remove the RIS from the rifle, even if all attachments have been removed prior to trying.

This may apply to all handguards; I was only able to test with the RIS.