Hey there! Thanks for bringing this to our attention. This is a gap we'll need to fix on our end -- I've filed the issue here:
https://github.com/medplum/medplum/issues/8843.
Current state is that the default patient access policy applies only when patients sign up via the open patient registration flow, not when they're invited via the admin interface. Invites through the admin interface only get an access policy applied if one is explicitly set on the invite (or added on ProjectMembership afterward), which is why all your invited patients had full access.