Thursday, August 14, 2025

Tradeoffs to Consider: Serving Model Predictions

Credit to Santiago over at ML.School with his course Building AI/ML Systems That Don't Suck for thinking through and sharing this image during his course. 

This is similar to other tradeoffs in business where it's important to shape, temper, and communicate your expectations. What are your priorities? How do you solve the business problem while juggling the quality, speed, and cost of the output? 

Spend the time upfront defining the business problem and the ideal solution. 

The image is self-explanatory. 


Posted by at

Wednesday, July 30, 2025

AI for Tech Pros in Seven Days: Comprehensive Lesson Plan

This is just one way to learn these tools. The most important thing isn't that you do everything here. The most important thing is that you start. Do something. Learn something. Move in the right direction. We're not ever looking for perfection. We're looking for incremental mastery of new ideas. 

You've got this. 

AI for Tech Pros: No-Code Mastery – Comprehensive Lesson Plan

This creates a comprehensive, self-contained lesson plan tailored for IT professionals like you—with years of tech experience but minimal coding background. The plan emphasizes no-code tools, leverages your IT intuition (e.g., troubleshooting workflows), and builds practical AI skills for real-world applications, like automating IT tasks.

The lesson plan is structured for 7 core days (plus Day 0 orientation), with each day capped at 4 hours to fit busy schedules. It's achievable over 1-2 weeks, assuming self-paced learning. Total estimated time: 30 hours. Focus on hands-on projects to reinforce concepts and use the pitfalls to proactively avoid common beginner frustrations.

Approach:

  • Target Audience: IT pros with tech experience but little coding—perfect for those who've managed systems or networks but want to add AI without development hurdles.
  • Prerequisites: Basic computer skills; familiarity with tools like Google Sheets. No coding required.
  • Goals: By the end, you'll build and deploy AI prototypes (e.g., agents, automations, MVPs) applicable to IT work, gaining confidence as an AI generalist.
  • Materials: Free-tier tools; a journal in Notion for notes and reflections.
  • Pacing Tips: Dedicate 4 hours/day; take breaks for testing. If needed, use optional extensions (1-2 days/week) for deeper practice on complex IT apps like predictive maintenance.
  • Support: Simulate "office hours" by reviewing pitfalls and journaling fixes. Join communities for peer help.
  • Assessment: Daily assignments build a portfolio; on Day 7, pitch your MVP to yourself.
  • Philosophy: AI as an extension of your IT toolkit—practical, no/low-code, and empowering.

Day 0: Orientation and Mindset Shift (2 hours)

Focus: Bridge your IT experience to AI—no code needed.
Activities: Review the full lesson plan; set personal goals (e.g., "Automate my daily IT reports"). Watch intro videos on AI basics to shift mindset from traditional IT to AI-enhanced workflows. Join no-code communities for ongoing support.
Resources and Links:

  1. Overloading on Tools Too Early: Don't install everything at once—focus on Day 1 needs first to avoid setup fatigue. (Tip: Prioritize Ollama if your machine has a GPU for smoother local runs.)
  2. Ignoring Account Sign-Ups: Free tiers require email verification; delays happen if you use a work email with strict filters. (Tip: Use a personal Gmail for quick access.)
  3. Underestimating Goal Setting: Vague goals lead to scattered focus—make them specific, like "Build an AI for IT ticket summaries." (Tip: Revisit your journal daily.)

Day 1: AI Fundamentals and Local Playground (4 hours)

Tailored Twist: Relate LLMs to IT databases—querying data without SQL.
Steps:

  1. Fundamentals (1 hour): Learn LLMs and transformers; watch intro resources.
  2. Prompt Engineering (1 hour): Practice techniques in OpenAI Playground.
  3. Deploy Models (1.5 hours): Use Ollama, Msty, and Bolt; test queries.
  4. Pipelines (30 mins): Chain prompts for complex outputs; take a 10-min break if needed.
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Hardware Mismatch for Local Models: Ollama may run slow on older CPUs—expect longer load times if no GPU. (Tip: Test with small models like "llama3:8b" first; switch to cloud if needed.)
  2. Poor Prompting Habits: Vague prompts yield junk outputs, like asking "Explain AI" without context. (Tip: Always specify role, format, and examples—e.g., "As an IT expert, explain LLMs in bullet points.")
  3. Installation Errors: Bolt or Msty might conflict with antivirus software. (Tip: Temporarily disable security during install; check tool forums for OS-specific fixes.)
  4. Forgetting Privacy: Local runs are great for sensitive IT data, but default to cloud for quick tests. (Tip: Avoid uploading proprietary info to OpenAI.)

Day 2: Media Generation and Clones (4 hours)

Tailored Twist: Use for IT visuals (e.g., generate diagrams for reports).
Steps:

  1. Images (1 hour): Prompt and refine in Midjourney.
  2. Videos (1 hour): Animate and edit in Runway and Veed.io.
  3. Voice Cloning (1 hour): Use ElevenLabs; test outputs.
  4. Integration (1 hour): Combine for a full clone; include short breaks between tests.
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Discord Overwhelm for Midjourney: As an IT pro, you might skip the bot commands—prompts fail without "/imagine". (Tip: Watch a 5-min Discord tutorial; start in a private server.)
  2. File Size Limits: Uploading large audio for ElevenLabs cloning hits free-tier caps quickly. (Tip: Trim samples to 30 seconds; use low-res for tests.)
  3. Inconsistent Outputs: AI media can vary wildly—e.g., clones sounding robotic if prompts lack detail. (Tip: Refine with specifics like "Natural, professional tone with pauses.")
  4. Integration Glitches: Veed.io exports may not play well with other tools. (Tip: Export in MP4; test compatibility early.)

Day 3: Automations for IT Workflows (4 hours)

Tailored Twist: Build expense trackers as "ticket automators" using your IT flow knowledge; extend to predictive IT tasks like ticket forecasting.
Steps:

  1. Intro (45 mins): Triggers and actions overview.
  2. Setup (1 hour): Create n8n workflow; compare with Zapier.
  3. Build Tracker (1.5 hours): Integrate AI for categorization using Make, Tally, and Akkio for predictions (e.g., forecast IT downtime).
  4. Modules (45 mins): Explore advanced features like loops; pause for testing.
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Connection Failures: n8n/Zapier integrations break if API keys expire or permissions are wrong—common in IT setups. (Tip: Double-check OAuth during setup; refresh tokens if errors occur.)
  2. Over-Automating Early: Trying complex flows before basics leads to loops that crash. (Tip: Start with 2-3 nodes; test incrementally, like in IT debugging.)
  3. Data Privacy Oversights: Automating with Google Sheets shares data—avoid sensitive IT info. (Tip: Use anonymous test data; enable 2FA on accounts.)
  4. Free-Tier Throttling: Zapier limits zap runs; exceed and workflows pause. (Tip: Monitor usage in the dashboard; opt for Make for more generous limits.)

Day 4: Building AI Agents (4 hours)

Tailored Twist: Agents as "smart assistants" for IT tasks (e.g., research troubleshooting or predictive alerts).
Steps:

  1. Basics (1 hour): Agents and autonomy concepts.
  2. Setup (1 hour): Configure tools in LangChain and AutoGPT.
  3. Build (1.5 hours): Research agent with multi-steps using CreateAI, AgentCloud, and Lindy AI for agentic workflows.
  4. Refine (30 mins): Add memory and testing; short break for debugging.
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Tool Overload: LangChain's no-code mode still feels "code-y"—non-coders skip dependencies. (Tip: Use pre-built templates; focus on AutoGPT for simpler starts.)
  2. Infinite Loops: Agents can loop on tasks if prompts aren't bounded, eating API credits. (Tip: Add "Stop after 5 steps" in instructions; monitor runs closely.)
  3. Memory Mishaps: Forgetting to enable agent memory leads to repetitive outputs. (Tip: Test with multi-turn queries; relate to IT caching concepts.)
  4. API Rate Limits: CreateAI/AgentCloud hits limits fast in free mode. (Tip: Space out tests; use local Ollama integration for offline practice.)

Day 5: Advanced Integrations with MCPs (4 hours)

Tailored Twist: MCPs for data fetching, like pulling IT logs into AI for analysis.
Steps:

  1. Intro (1 hour): MCP concepts from docs.
  2. Build (1 hour): Personas in Claude and Perplexity.
  3. Advanced (1.5 hours): Micro apps and servers with Apify, Kite, VAPI.
  4. Test (30 mins): Generate a stylized report; break for verification.
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Context Overload: Uploading too much data to Claude crashes sessions—common for IT folks with big files. (Tip: Chunk data; start with small tests.)
  2. Misconfigured Servers: MCP servers fail if ports conflict with IT firewalls. (Tip: Use default settings; check tool docs for port tweaks.)
  3. Persona Inconsistencies: Vague MCP definitions lead to off-topic responses. (Tip: Define strict roles, like "IT Analyst summarizing logs.")
  4. Scraping Limits: Apify hits rate limits on web data. (Tip: Use sparingly; cache results in Notion for reuse.)

Day 6: Voice Agents and Tech Deep Dive (4 hours)

Tailored Twist: Voice bots for IT support (e.g., querying knowledge bases hands-free or alerting on predictions).
Steps:

  1. Tech 101 (45 mins): APIs and embeddings overview.
  2. Basics (1 hour): VAPI setup with advanced prompting; incorporate CodeLlama.
  3. Build (1.5 hours): Bot with transcription via Whisper and functions.
  4. Deploy (45 mins): Test and refine conversations; take breaks between calls.
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Audio Quality Issues: Poor mic input makes Whisper transcription inaccurate—echoes your IT audio troubleshooting. (Tip: Use a quiet room; test with clear speech.)
  2. Prompt Latency: Overly complex prompts slow VAPI responses. (Tip: Keep system prompts under 200 words; optimize like IT query optimization.)
  3. Integration Gaps: CodeLlama for meta-prompting fails without proper imports (even no-code). (Tip: Copy from tutorials; fallback to basic prompting.)
  4. Free-Tier Call Limits: VAPI caps voice minutes quickly. (Tip: Script short tests; record offline for practice.)

Day 7: MVP Build and Capstone (4 hours)

Tailored Twist: Build an IT-focused MVP (e.g., automated dashboard with predictions).
Steps:

  1. Ideate (45 mins): Brainstorm ideas.
  2. Design (1 hour): Sketch in Framer.
  3. Build Jerry (1 hour): Agent with embeddings using Softr and Tally.
  4. MVP and Ship (1.15 hours): Automate features with Bubble and Make; deploy and demo. Complete self-certification (e.g., Google AI Essentials badge).
    Resources and Links:

Common Pitfalls/Gotchas:

  1. Scope Creep: Adding too many features mid-build crashes no-code apps like Bubble. (Tip: Stick to 3 core functions; iterate post-MVP.)
  2. Deployment Hiccups: Softr/Framer previews work but live deploys fail on custom domains. (Tip: Use free subdomains; test links immediately.)
  3. Embedding Overkill: Misusing vector embeddings bogs down performance. (Tip: Only add if needed for search; keep simple for your first MVP.)
  4. Pitch Neglect: Forgetting to document makes reflection hard. (Tip: Record a 1-min video; tie back to IT goals.)

Additional Learning Resources

To continue your journey beyond this course, here's an updated, curated list of resources focused on no-code and low-code AI for IT professionals, with an added emphasis on code-based tools integrated with Visual Studio Code (VS Code). These resources extend the no-code theme of the course while providing a gradual bridge to low-code and code-based AI development, tailored for IT pros with minimal coding experience but strong tech intuition. Prioritize based on your goals—start with no-code/low-code for immediate wins, then explore VS Code tools for advanced projects. All resources are selected for relevance to 2025 trends (e.g., agentic AI, predictive analytics) and accessibility (free or freemium tiers).

No-Code Tools (2025 Recommendations)

These build on course tools for IT applications like predictive analytics, automation, and app building. Free tiers available; focus on drag-and-drop interfaces.

Low-Code Tools

These offer visual interfaces with minimal coding, ideal for IT pros transitioning from no-code to light scripting, often compatible with VS Code for configuration.

Code-Based Tools (Integrated with VS Code)

These are beginner-friendly for IT pros ready to explore coding, leveraging VS Code as a lightweight IDE. VS Code extensions simplify AI development.

  • Visual Studio Code (VS Code): Free, open-source IDE for AI scripting and tool integration: https://code.visualstudio.com/.
    • Why Use? VS Code supports Python, JavaScript, and no-code/low-code extensions, making it ideal for IT pros experimenting with AI libraries.
    • Setup Tip: Install extensions like Python, Jupyter, and GitHub Copilot for AI assistance.
  • Hugging Face Transformers: Open-source AI library for LLMs, usable in VS Code with Python: https://huggingface.co/docs/transformers/index.
    • VS Code Integration: Use the Hugging Face extension (search in VS Code marketplace) for model management.
  • TensorFlow.js: JavaScript-based ML library for browser-based AI, runs in VS Code: https://www.tensorflow.org/js.
    • VS Code Integration: Use JavaScript extensions and Live Server for testing.
  • PyTorch: Open-source ML framework, beginner-friendly with VS Code Python support: https://pytorch.org/.
    • VS Code Integration: Install PyTorch via pip in VS Code’s terminal; use Jupyter notebooks for experiments.
  • LangChain.js: JavaScript version of LangChain for agentic AI, usable in VS Code: https://js.langchain.com/.
    • VS Code Integration: Use Node.js extension for scripting; pair with LangChain templates.
  • Ollama Extension for VS Code: Run local models directly in VS Code: https://marketplace.visualstudio.com/items?itemName=ollama.ollama.
    • Why Use? Extends course’s Ollama usage with a familiar IDE interface.

Free/Open-Source Courses

These are beginner-friendly, no-code/low-code focused, with certificates where possible. Some include VS Code for light coding.

YouTube Videos and Channels

Visual, tutorial-based learning for no-code, low-code, and VS Code-integrated AI. Subscribe for ongoing updates.

Advanced Resources

For next-level learning once comfortable with no-code/low-code, including VS Code workflows.

Learning Path Suggestions

  1. No-Code First: Start with Elements of AI and Akkio for immediate IT applications.
  2. Low-Code Transition: Try AppGyver or Power Automate, using visual scripting to ease into coding concepts.
  3. VS Code Exploration: Install VS Code with Python/Jupyter extensions; follow Sentdex or fast.ai tutorials for light scripting (e.g., simple LLM queries).
  4. Community Engagement: Share projects on NoCode Founders or Reddit; ask for feedback on X.
  5. Certification: Complete Google AI Essentials or MIT’s 6.S191 for a badge to boost your IT resume.

These resources are current for July 2025, emphasizing no-code/low-code with a clear path to VS Code for IT pros ready to experiment.

 

Posted by at

Saturday, July 12, 2025

A visual introduction to ML | Tuning | Bias-Variance

I came across R2D3's interactive guide on machine learning basics (Parts 1 & 2) and thought it'd be useful to share. It's a visual explanation using a dataset of homes in San Francisco vs. New York for classification.

Part 1: Basics of ML and Decision Trees

  • ML uses statistical techniques to identify patterns in data for predictions, e.g., classifying homes by features like elevation and price per sq ft.
  • Decision trees create boundaries via if-then splits (forks) on variables, recursively building branches until patterns emerge.
  • Training involves growing the tree for accuracy on known data, but overfitting can occur, leading to poor performance on unseen test data.

Part 2: Bias-Variance Tradeoff

  • Models have tunable parameters (e.g., minimum node size) to control complexity.
  • High bias: Overly simple models (e.g., a single-split "stump") ignore nuances, causing systematic errors.
  • High variance: Overly complex models overfit to training data quirks, causing inconsistent errors on new data.
  • Optimal models balance bias and variance to minimize total error; deeper trees reduce bias but increase variance.

Created by Stephanie Yee (statistician) and Tony Chu (designer) at R2D3.us. Great for intuitive understanding—check it out if interested.

Sources: 

Posted by at

Tuesday, June 24, 2025

Reducing Memory-Related Vulnerabilities - NSA New Guidance

 

Here's the summary of the new Memory Safe Language (MSL) guidance. Why do you care? Memory safety vulnerabilities persist at alarming rates. 

Some older stats: 

  • About 70% of Microsoft CVEs are memory safety issues
  • 70% of Google Chromium project vulnerabilities are memory safety related
  • 67% of zero-day vulnerabilities in 2021 were memory safety issues
Some newer stats. Because there's still a challenge.
  • 66% of iOS CVEs are memory safety related
  • 71% of macOS CVEs stem from memory safety issues

If you haven't been thinking about root cause analysis for reducing software vulnerabilities you're already behind your peers. And here is a major root cause. Memory Safe Programming Languages (MSLs) can eliminate these vulnerabilities entirely. These are programming languages designed to prevent common memory-related coding errors that malicious actors routinely exploit.

Business and Technical Benefits

All of this is interesting... but take note: 

Security Benefits: (obviously...)

  • Vulnerability Elimination: Entire classes of bugs become impossible
  • Reduced Attack Surface: Forces attackers to find other types of vulnerabilities
  • Proactive Protection: Prevents problems during development rather than patching them later

Reliability Benefits: (good for business...)

  • Fewer Crashes: Programs behave more predictably
  • Better Error Messages: When problems occur, MSLs provide clearer debugging information
  • Increased Uptime: More stable systems mean less downtime

Productivity Benefits: (good for the people...)

  • Faster Debugging: Developers spend less time hunting memory bugs
  • Focus on Features: Teams can concentrate on building functionality instead of fixing memory issues
  • Reduced Emergency Patches: Fewer urgent security updates needed

Sources:


Posted by at

Monday, April 7, 2025

Databricks AI Security Framework (DASF) | Third-party Tools

Great work - Amazing work - by the team at Databricks. Nice job!

Databricks AI Security Framework (DASF) | Databricks

This link leads to a PDF that selflessly has links to a LOT of information. Thank you for including them!

Here's one such list. I'm storing it here as a quick yellow sticky. Go check out their work for more. 

Tool Category

Tool URL

Tool Description

Model Scanners

HiddenLayer Model Scanner

A tool that scans AI models to detect embedded malicious code, vulnerabilities, and integrity issues, ensuring secure deployment.

Fickling

An open-source utility for analyzing and modifying Python pickle files, commonly used for serializing machine learning models.

Protect AI Guardian

An enterprise-level tool that scans third-party and proprietary models for security threats before deployment, enforcing model security policies.

AppSOC’s AI Security Testing solution

AppSOC’s AI Security Testing solution helps in proactively identifying, and assessing the risks from LLM models by automating model scanning, simulating adversarial attacks, and validating trust in connected systems, ensuring the models and ecosystems are safe, compliant, and deployment-ready

Model Validation Tools

Robust Intelligence Continuous Validation

A platform offering continuous validation of AI models to detect and mitigate vulnerabilities, ensuring robust and secure AI deployments.

Protect AI Recon

A product that automatically validates LLM Model performance across common industry framework requirements (OWASP, MITRE/ATLAS).

Vigil LLM security scanner

A tool designed to scan large language models (LLMs) for security vulnerabilities, ensuring safe deployment and usage.

Garak Automated Scanning

An automated system that scans AI models for potential security threats, focusing on detecting malicious code and vulnerabilities.

HiddenLayer AIDR

A solution that monitors AI models in real time to detect and respond to adversarial attacks, safeguarding AI assets.

Citadel Lens

A security tool that provides visibility into AI models, detecting vulnerabilities and ensuring compliance with security standards.

AppSOC’s AI Security Testing solution

AppSOC’s AI Security Testing solution helps in proactively identifying, and assessing the risks from LLM models by automating model scanning, simulating adversarial attacks, and validating trust in connected systems, ensuring the models and ecosystems are safe, compliant, and deployment-ready

AI Agents

Arhasi R.A.P.I.D

A platform offering rapid assessment and protection of AI deployments, focusing on identifying and mitigating security risks.

Guardrails for LLMs

NeMo Guardrails

A toolkit for adding programmable guardrails to AI models, ensuring they operate within defined safety and ethical boundaries.

Guradrails AI

A framework that integrates safety protocols into AI models, preventing them from generating harmful or biased outputs.

Lakera Guard

A security solution that monitors AI models for adversarial attacks and vulnerabilities, providing real-time protection.

Robust Intelligence AI Firewall

A protective layer that shields AI models from adversarial inputs and attacks.

Protect AI Layer

Layer provides LLM runtime security including observability, monitoring, blocking for AI Applications. The enterprise grade offering brought to you by the same team that built the industry leading open source solution LLM Guard.

Arthur Shield

A monitoring solution that tracks AI model performance and security, detecting anomalies and potential threats in real time.

Amazon Guardrails

A set of safety protocols integrated into Amazon's AI services to ensure models operate within ethical and secure boundaries.

Meta Llama Guard

Meta implemented security measures to protect their Llama models from vulnerabilities and adversarial attacks.

Arhasi R.A.P.I.D

A platform offering rapid assessment and protection of AI deployments, focusing on identifying and mitigating security risks.

DASF Validation and Assessment Products and Services

Safe Security

SAFE One makes cybersecurity an accelerator to the business by delivering the industry's only data-driven, unified platform for managing all your first-party and third-party cyber risks.

Obsidian

Obsidian Security combines application posture with identity and data security, safeguarding SaaS.

EQTY Labs

EQTY Lab builds advanced governance solutions to evolve trust in AI.

AppSOC

Makes Databricks the most secure AI platform with real-time visibility, guardrails, and protection.

Public AI Red Teaming Tools

Garak

An automated scanning tool that analyzes AI models for potential security threats, focusing on detecting malicious code and vulnerabilities.

Protect AI Recon

A product with a full suite of Red Teaming options for AI applications, including a library of common attacks, human augmented attacks, and LLM generated scans; complete with mapping to common industry frameworks like OWASP and MITRE/ATLAS.

PyRIT

A Python-based tool for testing the robustness of AI models against adversarial attacks, ensuring model resilience.

Adversarial Robustness Toolbox (ART)

An open-source library that provides tools to assess and improve the robustness of machine learning models against adversarial threats.

Counterfeit

A tool designed to test AI models for vulnerabilities by simulating adversarial attacks, helping developers enhance model security.

ToolBench

A suite of tools for evaluating and improving the security and robustness of AI models, focusing on detecting vulnerabilities.

Giskard-AI llm scan

A tool that scans large language models for security vulnerabilities, ensuring safe deployment and usage.

Hidden Layer - Automated Red Teaming for AI

A service that simulates adversarial attacks on AI models to identify vulnerabilities and strengthen defenses.

Fickle scanning tools

Utilities designed to analyze and modify serialized Python objects, commonly used in machine learning models, to detect and mitigate security risks.

CyberSecEval 3

A platform that evaluates the security posture of AI systems, identifying vulnerabilities and providing recommendations for mitigation.

Parley

A tool that facilitates secure and compliant interactions between AI models and users, ensuring adherence to safety protocols.

BITE

A framework for testing the security and robustness of AI models by simulating various adversarial attack scenarios.

Purple Llama

Purple Llama is an umbrella project that over time will bring together tools and evals to help the community build responsibly with open generative AI models. The initial release will include tools and evals for Cyber Security and Input/Output safeguards but we plan to contribute more in the near future.

 


Posted by at
Subscribe to: Posts (Atom)