Output Monitoring to Overcome LLM Vulnerabilities