If the servers restart mid-game play, if you get booted, etc... something is thrown off with XBOX. The ability to "Invite to Game" is greyed out with friends/party, as well as "Join Game" is greyed out when you highlight your friends.
To fix this, you have to terminate the game on XBOX (Go to Xbox Menu --> Highlight DayZ --> Menu Button --> Quit).
Restart the game, join a server, and it all comes back. As you hop between servers on your own (using Menu --> Exit in DayZ) it remains working. But once you get booted or a server reboots, it greys it all out.