I built a small RAG (Retrieval Augmented Generation) project where a user can ask questions from a PDF, and the LLM answers from that PDF along with the page number to look at. The stack is LangChain, OpenAI embeddings, and Qdrant running in Docker.

A small note before we start: this exact same pipeline is what powers web-apps like an “AI Tutor in Educative”, an “AI web page builder”. The only thing that changes between those products and my PDF Q&A is the data source. That is the key idea to take away.

What RAG is, in one line

Take a document → break it into small chunks → turn each chunk into a vector (a list of numbers) → store those vectors in a database. Later, when the user asks a question, turn the question into a vector too, find the closest chunks, and feed them to an LLM as context.

INDEXING (run once)
───────────────────

   PDF file
      │
      ▼
   PyPDFLoader
      │  one Document per page (text + metadata)
      ▼
   RecursiveCharacterTextSplitter
      │  chunk_size = 1000, overlap = 200
      ▼
   ~200 small text chunks
      │
      ▼
   OpenAIEmbeddings (text-embedding-3-small)
      │  each chunk becomes a 1536-dim vector
      ▼
   Qdrant (running in Docker)
      │  vector + chunk text + page metadata stored


QUERY (every user question)
───────────────────────────

   "explain me about variables"
      │
      ▼
   OpenAIEmbeddings
      │  same model, same vector space
      ▼
   Query vector (1536-dim)
      │
      ▼
   Qdrant similarity_search (cosine, top k = 4)
      │  closest 4 chunks come back
      ▼
   Build a prompt with those chunks as context
      │
      ▼
   OpenAI Chat Completion (gpt-5-nano)
      │
      ▼
   Final answer + page citations

The stack and why I picked it

• LangChain is the glue. It gives me one interface to talk to many providers. If I want to swap OpenAI for Cohere, or Qdrant for Pinecone, I change one line. It also has loaders for PDFs, websites, Notion, Google Docs, CSVs, image files, and a lot more. Plus tools for chains, agents, memory, prompts, and output parsing.
• Qdrant is the vector database. I ran it locally in Docker so I don’t have to pay for a managed service while learning.
• OpenAI text-embedding-3-small is the embedding model. More on why this one and not the large one further down.
• uv instead of pip. Faster, lockfile-based, and the modern Python experience.

Some things worth pointing out:

• The embedding model is just a client — no API call happens when you create it. The actual call to OpenAI happens inside from_documents.
• from_documents is the convenience method. It connects to Qdrant, creates the collection, embeds every chunk, and inserts it. One call, three jobs.

Now the real question — how big a PDF can I actually ingest?

This is the part I couldn’t find a straight answer to in any tutorial. The pipeline above worked beautifully for my 123 KB, 71-page PDF. But what about a 100 MB book? A 1 GB legal document dump?

The answer has many layers. There is no single PDF size limit.

The actual bottleneck is almost always OpenAI, not Python or Qdrant.

What OpenAI actually limits

There are three things to watch on the embeddings API.

1. Max tokens per single input — 8,191

Each chunk you send to be embedded can be at most 8,191 tokens, which is roughly 32,000 characters. My chunk_size=1000 is way below that, so this is never hit in normal RAG. Official source: https://platform.openai.com/docs/guides/embeddings

2. Rate limits — RPM (requests per minute) and TPM (tokens per minute)

These depend on your account tier. Tier 1, where most new accounts start, looks like this:

Your tier moves up automatically as you spend more on the platform.

• Official rate limit docs: https://platform.openai.com/docs/guides/rate-limits/usage-tiers
• Your own account’s limits: https://platform.openai.com/account/limits

When you hit the TPM, LangChain backs off and retries automatically. So your script doesn’t crash — it just takes longer.

3. The cost — $0.02 per 1M tokens for text-embedding-3-small

This is the most underrated number. A 100 MB text PDF is roughly 20–50 million tokens. That works out to about $0.40 to $1.00 in embedding cost. Real-world cheap. Pricing page: https://platform.openai.com/docs/pricing

Most developers and IT beginners only touch networking when something breaks.

The internet is “working,” the API is responding, Docker containers are starting, and life moves on.

Until one day:

• Your app cannot connect to a database
• Docker says the daemon is unreachable
• A server responds slowly for no obvious reason
• A device disappears from the network
• SSH refuses connections
• Your Raspberry Pi suddenly becomes invisible

That is usually when people realize they do not actually understand their network.

One of the best ways to fix that is by learning how to inspect and troubleshoot networks directly from the terminal using PowerShell and TCP-based tools.

This post is inspired by a networking article from Paessler Blog and expands it into a more developer-focused practical guide. (blog.paessler.com)

Why PowerShell Matters for Networking

For years, networking on Windows relied heavily on old tools like:

• ping
• ipconfig
• netstat
• tracert
• netsh

These still work.

But modern Windows networking is increasingly PowerShell-driven. Microsoft has gradually shifted many administration and diagnostic workflows toward PowerShell because it is:

• scriptable
• automatable
• object-oriented
• easier to integrate into larger workflows

Instead of parsing messy text output, PowerShell returns structured objects you can filter, sort, and automate. (blog.paessler.com)

Understanding TCP Before Troubleshooting

Most developer tools rely on TCP underneath:

• Docker
• APIs
• databases
• SSH
• web servers
• cloud platforms

TCP (Transmission Control Protocol) is responsible for:

• reliable communication
• packet ordering
• retransmission
• connection establishment

When applications fail to communicate, the problem is often not the application itself — it is the TCP/network layer.

Understanding a few networking commands can save hours of frustration.

1. Check Your Network Configuration

The first thing to inspect is your machine’s network configuration.

Get-NetIPAddress

This gives you:

• IP address
• subnet
• interface
• IPv4/IPv6 details

You can also use:

ipconfig

But PowerShell gives more structured information.

2. Test Connectivity Properly

Most people know ping.

PowerShell modernizes it with:

Test-Connection google.com

You can test multiple systems:

Test-Connection google.com, github.com

Or continuously monitor latency:

Test-Connection 192.168.1.1 -Count 10

This helps detect:

• unstable Wi-Fi
• packet loss
• intermittent routing issues

Microsoft’s networking documentation also recommends PowerShell-based connectivity testing for modern administration workflows. (Microsoft Learn)

3. Discover Devices on Your Network

One of the most useful troubleshooting techniques is identifying what devices are actually online.

You can inspect your ARP table:

arp -a

This maps:

• IP addresses
• MAC addresses

to devices your machine has communicated with recently.

Modern network discovery guides still rely heavily on ARP-based discovery for local troubleshooting. (blog.paessler.com)

4. Scan an Entire Subnet

Suppose your Raspberry Pi disappeared from the network.

You can sweep your subnet:

1..254 | ForEach-Object {
    Test-Connection "192.168.1.$_" -Count 1 -Quiet
}

This checks which devices respond.

Microsoft demonstrates similar subnet enumeration approaches in PowerShell networking examples. (Microsoft Learn)

5. Find Open TCP Ports

Sometimes the device is online, but the service is not.

For example:

• SSH may not be running
• Docker daemon may be down
• database service may have crashed

You can test ports directly:

Test-NetConnection 192.168.1.10 -Port 22

Example ports:

This immediately tells you whether the issue is:

• routing
• firewall
• service availability
• TCP-level rejection

6. Monitor Active TCP Connections

To inspect live connections:

Get-NetTCPConnection

You can filter by state:

Get-NetTCPConnection | Where-Object {$_.State -eq "Established"}

Or inspect a specific port:

Get-NetTCPConnection -LocalPort 80

This is extremely useful when debugging:

• APIs
• containers
• backend services
• reverse proxies

7. Troubleshoot Docker Connectivity

If Docker suddenly breaks on Windows, PowerShell networking commands become incredibly useful.

For example, errors like:

failed to connect to the docker API

often indicate:

• Docker daemon stopped
• named pipe failure
• TCP bridge issue
• virtualization problem

Useful checks:

Get-Service *docker*

Test-NetConnection localhost -Port 2375

docker context ls

Understanding networking basics makes Docker troubleshooting much easier because Docker heavily relies on virtual networking and TCP communication internally.

8. Automate Network Diagnostics

This is where PowerShell becomes powerful.

Instead of manually checking systems one by one, you can automate diagnostics.

Example:

$servers = @(
    "192.168.1.10",
    "192.168.1.11",
    "192.168.1.12"
)

foreach ($server in $servers) {
    Test-NetConnection $server -Port 22
}

Now you can verify SSH availability across multiple systems automatically.

This scales extremely well for:

• labs
• classrooms
• hubs
• server farms
• IoT deployments

9. Network Monitoring at Scale

At some point, command-line troubleshooting is not enough.

That is where monitoring systems come in.

Platforms like PRTG Network Monitor automate:

• network discovery
• latency tracking
• device monitoring
• traffic inspection
• uptime monitoring

Modern network visibility systems combine:

• ARP discovery
• TCP monitoring
• SNMP
• device inventories
• automated alerting

to give administrators full infrastructure visibility. (blog.paessler.com)

Real Lesson: Developers Need Networking Skills

One pattern keeps repeating in modern software engineering:

The deeper you go, the more networking matters.

Machine learning engineers, backend developers, DevOps engineers, cybersecurity analysts, embedded engineers, and cloud architects all eventually run into:

• ports
• sockets
• firewalls
• routing
• DNS
• TCP failures

Networking knowledge is no longer optional.

Final Thoughts

Learning PowerShell networking commands is not about becoming a network engineer overnight.

It is about gaining visibility.

Once you can inspect:

• connections
• devices
• ports
• adapters
• latency
• routing

you stop guessing and start diagnosing systems properly.

And honestly, that is one of the biggest transitions from beginner developer to advanced engineer.

Further Reading

• Paessler Networking Article
• Microsoft PowerShell Networking Tasks
• PowerShell Network Troubleshooting Guide

(blog.paessler.com)