Yapay Zeka

The Creator Void: My Journey from Machines to Code

I have always had an inner urge from my childhood; I have been a curious kid from early. I vividly remember figuring things out on my own—how phones work, how machines are built. Every opportunity to disassemble a machine was a golden chance. I questioned things, I tried things, and I failed at many, of course. But importantly, I learned how things work, and that has always been a life thread for me.

I think of myself as intellectually thirsty, with a “creator void” inside me. I’m not satisfied with mere attainments. However, there have been sidetracks and “sideshows,” as my dad would call them, that have kept me from reaching the ultimate version of me—but today, that would not be our topic.

The Barrier of the “Qualified IT Guy”

Growing up in my teens, my passion with gadgets did not diminish but became augmented. I started opening up CPUs, disassembling the CMOS and the hard drives, reassembling them, and installing operating systems. But I dreaded one thing: writing lines of code. I never thought I was worthy or qualified. I always looked to a point in the future when my brain would be able to comprehend such things.

Fast forward to my campus days doing my Bachelor’s in Business and IT. I had started out hot, but again, to use my dad’s words, I became sidetracked. I abandoned the journey for a long time—so long that I graduated without being a “qualified IT guy.” But life granted me a chance once again to rediscover the ultimate me. For the past two months, I have written more lines of code than I have written my entire life, and there is something I’m learning from all this that I hope helps software engineer aspirants.

1. The House and the Foundation

Just as building a house requires a solid foundation, a software engineer needs to be grounded on elementary principles of writing code. Learn how data is manipulated at the very core—strings manipulation, bytes, bits, ASCII. Learn the foundational blocks and know them very well. It can take up to a month of consistency just practicing the basics, but later, as you start working on projects like building simple tools, you will realize that you are just applying fundamental blocks but now compiling them.

Think of it like building a kitchen: you will use many different blocks, but you design it in a way that serves the purpose of cooking. I insist highly on being grounded on the building blocks.

2. The Freedom of the Disciplined

Secondly, to code requires a high frequency of consistency. You must make sure that you practice almost daily; you make it a part of your life. There is no other way to scale up. You must be consistent and disciplined. As the great Eliud Kipchoge would say: “Only the disciplined ones are free in life. If you are undisciplined, you are a slave to your moods and your passions.” You might be very talented, but all that talent will avail to nothing if you are not disciplined in life. As it is written in Ecclesiastes 9:11: “I returned, and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favor to men of skill; but time and chance happens to them all.”

3. The Magic of Peer-to-Peer Learning

Thirdly, I would highly recommend peer-to-peer learning. It’s good to spend time alone trying to figure things out, but it should not end at that. There is a magic that sparks when you unite with like-minded peers. Sharing your ideas makes you understand them better.

Most importantly, you get to see things from a different light and a different perspective when others share their ideas and approach to problem-solving. Your understanding broadens. There is always a “good leaf” to borrow from others. As Charles Hazlitt Puffer wrote: “He who does not read the thoughts of others will never have any of his own.” It is just as it is put in Proverbs 27:17: “Iron sharpens iron; so a man sharpens the countenance of his friend.”

well, the journey has not been straight, there have always been crooked and thorny paths and to you, i do not promise a smooth journey, but a sure arrival.

Onelingo MVP: Lo que aprendí cuando mi app de idiomas no salió como esperaba

¡Hola a todos! Hoy no vengo a traerles el típico tutorial donde todo sale perfecto a la primera. Vengo a contarles la historia de Onelingo, una app de micro-desafíos diarios de idiomas que intenté construir usando Hugging Face Spaces.

Aunque el resultado final no fue exactamente lo que visualicé inicialmente, el proceso me dejó lecciones invaluables sobre despliegue, autenticación y la importancia de una arquitectura limpia. Aquí les comparto mi “Post-Mortem” técnico.

1. El concepto: Onelingo

La idea era crear una alternativa a Duolingo pero sin niveles infinitos: solo 3 micro-desafíos diarios (Reading, Listening y Speaking).

El stack elegido:

• Hosting: Hugging Face Spaces (Static SDK).
• IA de apoyo: HuggingChat (Llama 3 / Qwen).
• Frontend: HTML/JS/CSS con Tailwind vía CDN.

2. El primer obstáculo: El “Rito de Iniciación” de Git

Al intentar mover el código desde el editor web de Hugging Face a mi entorno local (WebStorm), me topé con el error que detiene a muchos principiantes:
remote: Invalid username or password. fatal: Authentication failed.

El aprendizaje:

Hugging Face no usa tu contraseña de usuario para las operaciones de Git. Necesitas un Access Token.

La solución técnica:

Si te sucede esto, debes forzar la URL del remoto para incluir tu token:

git remote set-url origin https://TU_USUARIO:[email protected]/spaces/TU_USUARIO/onelingo-mvp

Tip: Genera tu token en Settings > Access Tokens con permisos de “Write”.

3. El error de lógica: El “Esqueleto” sin cerebro

inicialmente, la app se veía genial con Tailwind, pero al hacer clic en las opciones de respuesta, no pasaba nada. El código generado por la IA era visualmente correcto pero funcionalmente vacío.

El Aprendizaje:

No confíes ciegamente en el primer bloque de código que te da una IA. Siempre verifica que las funciones de validación de estado (como el manejo de respuestas correctas/incorrectas) estén implementadas.

La solución futura:

Para evitar esto, mi recomendación es separar la lógica en archivos distintos desde el principio:

• index.html: Estructura.
• style.css: Diseño.
• app.js: El “cerebro” (validación de desafíos y Web Speech API).

4. Implementando speaking: El reto del Navegador

Intentamos usar la Web Speech API para el reconocimiento de voz. Aunque funciona bien en Chrome de escritorio, puede ser muy inestable en navegadores móviles o integrados (como el de Instagram o Twitter).

Aprendizaje clave:

Si tu app depende del micrófono, asegúrate de manejar los errores de permisos y de informar al usuario si su navegador no es compatible.

5. ¿Qué haría diferente la próxima vez?

Si estás pensando en crear un MVP similar, aquí te dejo mi hoja de ruta mejorada:

1. Arquitectura modular: No metas todo en un solo index.html. Usa una estructura de carpetas clara para que sea fácil de depurar.
2. Validación de IA: En lugar de pedir “hazme la app”, pide “hazme la lógica de validación para un array de objetos JSON”. Es mucho más preciso.
3. Hugging Face vs Vercel: Hugging Face es increíble para demos de IA, pero si tu app es puramente frontend (Static), Vercel o Netlify ofrecen un flujo de Git mucho más amigable para principiantes.

Conclusión

A veces, el éxito de un proyecto no está en el despliegue final, sino en los bugs que lograste entender. Onelingo me enseñó que la autenticación con tokens y la estructura de archivos son los cimientos de cualquier proyecto serio.

¿Has tenido algún proyecto que no salió como esperabas? ¡Cuéntame tu experiencia en los comentarios y qué aprendiste de ello! 🚀

Nota: Este tutorial es parte de mi serie de aprendizaje continuo como desarrolladora nómada. ¡Sigamos construyendo!

You’ve likely experienced the “Monday Morning Surprise.” You check your database after a weekend of automated scraping only to find thousands of new rows where the price column is empty, the product_name is “None,” and the stock_count is zero.

The script didn’t crash, your proxies worked perfectly, and the status codes were all 200 OK. But because the website owner changed a single CSS class from .price-value to .item-price, your scraper spent 48 hours collecting digital garbage.

In the world of Scraper Reliability Engineering (SRE), this is a silent failure. While traditional error handling focuses on network stability, Pydantic allows you to treat scraped data like a strict API contract. This approach detects layout changes the second they happen, ensuring your data pipeline remains untainted.

Why try/except Isn’t Enough

Most developers write defensive scrapers, wrapping extraction logic in try/except blocks to prevent the process from crashing when a single element is missing. While this keeps the script running, it often leads to silent data corruption.

Consider this common pattern:

def extract_product(element):
    try:
        return {
            "name": element.query_selector(".title").inner_text(),
            "price": float(element.query_selector(".price").inner_text().replace("$", ""))
        }
    except Exception:
        return None # The silent killer

If the .price selector fails, this function returns None. The main loop sees the None, skips the entry, and continues. You’ve lost data and won’t know why until you manually inspect the logs days later. You need to move away from “just don’t crash” and toward “fail fast when the contract is broken.”

Step 1: Defining the Data Contract with Pydantic

To build a reliable scraper, you must first define what “good data” looks like. Pydantic is a data validation library that uses Python type hints to enforce schemas. Instead of working with raw dictionaries, you map scraped data to a BaseModel.

Using Product Hunt as an example, we can define a model for the product name, upvote count, and primary tags.

from pydantic import BaseModel, Field
from typing import List

class ProductHuntItem(BaseModel):
    name: str = Field(min_length=1)
    upvotes: int = Field(ge=0) # Must be an integer >= 0
    tags: List[str]
    rank: int

By defining this model, you’ve created a data contract. If you try to pass a string like “1.2k” into upvotes, which expects an int, Pydantic will immediately raise a ValidationError.

Step 2: Integrating Validation with Playwright

When integrating this into a scraper using Playwright, the goal is to extract raw data into a dictionary and immediately attempt to instantiate the Pydantic model.

import asyncio
from playwright.async_api import async_playwright
from pydantic import ValidationError

async def scrape_product_hunt():
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        await page.goto("https://www.producthunt.com/")

        products = await page.query_selector_all('[data-test="post-item"]')
        valid_items = []

        for index, product in enumerate(products[:10]):
            try:
                # 1. Extract raw data
                raw_data = {
                    "name": await product.query_selector('[data-test="post-name"]').inner_text(),
                    "upvotes": await product.query_selector('[data-test="vote-button"]').inner_text(),
                    "tags": [await product.query_selector('[data-test="post-tag"]').inner_text()],
                    "rank": index + 1
                }

                # 2. Validate with Pydantic
                item = ProductHuntItem(**raw_data)
                valid_items.append(item)

            except ValidationError as e:
                print(f"Layout Change Detected at item {index}: {e.json()}")
            except Exception as e:
                print(f"Generic extraction error: {e}")

        await browser.close()
        return valid_items

if __name__ == "__main__":
    asyncio.run(scrape_product_hunt())

In this workflow, you extract data as a “dirty” dictionary first. When you call ProductHuntItem(**raw_data), Pydantic performs the type checking. If the website changes its structure, such as the upvote button containing text like “Free” instead of a number, the script catches it immediately.

Step 3: Catching the Drift

Imagine Product Hunt updates their UI. The upvote count, which used to be a simple integer in the HTML, is now formatted as “2,450”.

A standard Python int() conversion would fail, and a basic scraper might just store None. Pydantic provides a clear error trace instead:

{
  "type": "int_parsing",
  "loc": ["upvotes"],
  "msg": "Input should be a valid integer, unable to parse string as an integer",
  "input": "2,450"
}

This is shift-left testing for data extraction. You catch the error at the point of ingestion rather than weeks later during analysis.

Step 4: Advanced Cleaning with Validators

Websites are messy. Sometimes a layout change doesn’t mean the data is gone, but that the format has shifted. Pydantic’s @field_validator allows you to fix minor issues automatically before validation.

This updated model handles commas in numbers and “k” suffixes:

from pydantic import BaseModel, Field, field_validator

class ProductHuntItem(BaseModel):
    name: str
    upvotes: int

    @field_validator('upvotes', mode='before')
    @classmethod
    def clean_upvotes(cls, v: any) -> int:
        if isinstance(v, str):
            v = v.replace(',', '').strip().lower()
            if 'k' in v:
                return int(float(v.replace('k', '')) * 1000)
            return int(v)
        return v

By using mode='before', you intercept the raw data before Pydantic tries to force it into an integer. This makes the scraper resilient to cosmetic changes while maintaining a strict floor for data quality. If the input is “Add to Cart,” the validator will fail and alert you to a total layout shift.

Step 5: The Error Budget

In production web scraping, 100% success is rarely possible. A single item might fail validation because of a unique ad placement or a weird edge case. You don’t want to wake up an engineer for one failed row.

Instead, implement an error budget. Track the ratio of validation failures to successful extractions.

class ScrapeSession:
    def __init__(self):
        self.success_count = 0
        self.failure_count = 0
        self.threshold = 0.15  # 15% error budget

    def log_result(self, success: bool):
        if success:
            self.success_count += 1
        else:
            self.failure_count += 1

    def check_reliability(self):
        total = self.success_count + self.failure_count
        if total == 0: return

        error_rate = self.failure_count / total
        if error_rate > self.threshold:
            self.trigger_alert(error_rate)

    def trigger_alert(self, rate):
        print(f"CRITICAL: Scraper reliability dropped to {1-rate:.2%}. Layout change likely.")

# Usage in loop
session = ScrapeSession()
try:
    ProductHuntItem(**raw_data)
    session.log_result(True)
except ValidationError:
    session.log_result(False)

session.check_reliability()

If 2% of items fail, it is likely noise. If 20% fail, the website has likely changed its DOM structure and your error budget is spent. This is the moment to trigger a Slack notification or a PagerDuty alert.

Summary

Moving from basic scripts to Scraper Reliability Engineering requires a mindset shift. By treating scraped data as a typed contract rather than a loose collection of strings, you protect downstream applications from garbage data.

• Fail Fast: Use Pydantic to catch layout changes during extraction.
• Define Contracts: Create models that represent the ideal state of your data.
• Automate Cleaning: Use Pydantic validators to handle predictable noise like currency symbols.
• Monitor Ratios: Use an error budget to distinguish between minor glitches and systemic layout changes.

Implementing these patterns ensures you spend less time cleaning broken databases and more time building features with data you can trust.