Tag: AI limitations

  • LLMs: They Know More Than They Let On (And That’s a Problem)

    ⋅

    Ng S.T. Chong

    In a fascinating new study titled “Inside-Out: Hidden Factual Knowledge in LLMs,” researchers have uncovered compelling evidence of a significant gap between what LLMs know internally and what they can express in their outputs. This phenomenon, termed “hidden knowledge,” has important implications for evaluating and improving AI systems. The Knowledge Paradox Consider this scenario: You…

  • Beyond Words: Assessing LLMs’ Ability to Process and Reason with Tabular Data in PDFs 

    ⋅

    Ng S.T. Chong

    Large language models (LLMs) excel at handling text, but they can stumble when it comes to answering questions about numerical data in tables. To explore this limitation, I tested how advanced models handle two common scenarios:  Scenario 1 – Multi-page Table in a PDF The task was to find employees earning between $70,000 and $100,000 (name and…