On Sunday, the Secret Service allowed Holly Jackson and Kelly Charley to go