Discovering the Learning Machine
How can machines learn to solve problems in unpredictable, real-world environments when traditional algorithms fall short?
Algorithmic approaches in artificial intelligence offer smart solutions for complex, repetitive problems. But what happens when the problem itself keeps changing, or when information is incomplete? The journey toward flexible, adaptive AI begins here.
Early AI systems like MYCIN could outperform junior doctors in diagnosis accuracy, thanks to their ability to encode expert knowledge into hundreds of rules.
All the algorithmic examples given earlier are associated with AI because they’re smart solutions that solve repetitive and well-delimited yet complex problems requiring intelligence. They require an architect who studies the problem and chooses the right algorithm to solve it. Problem changes, mutations, or unusual characteristic displays can become a real problem for a successful execution of the algorithm. This is because learning the problem and its solution occurs once when you train the algorithm. For instance, you can safely program an AI to solve Sudoku puzzles. You can even provide flexibility that allows the algorithm to accept more rules or larger boards later.
Unfortunately, not all problems can rely on a Sudoku-like solution. Real-life problems are never set in simple worlds of perfect information and well-defined action. Consider the problem of finding a fraudster cheating on insurance claims or the problem of diagnosing a medical disease. You have to contend with these factors:
- A large set of rules and possibilities: The number of possible frauds is incredibly high; many diseases have similar symptoms.
- Missing information: Fraudsters can conceal information; doctors often rely on incomplete information (examinations may be missing).
- Problem rules aren’t immutable: Fraudsters discover new ways to arrange swindles or frauds; new diseases arise or are discovered.
To solve such problems, you can’t use a predetermined approach; instead, you need a flexible approach and must accumulate useful knowledge to face any new challenge. In other words, you continue learning, as humans do throughout their lives to cope with a changing and challenging environment.
- Smart algorithms excel at well-defined, repetitive tasks.
- Real-world problems often demand flexibility and ongoing learning.
Leveraging expert systems
Expert systems, or systems that use rules to make decisions, were the first attempt to escape the realm of hard-coded algorithms and create more flexible and smart ways to solve real-life problems. The idea at the core of expert systems was simple and well-suited at a time when storing and dealing with lots of data in computer memory was still costly. It may sound strange today, but in the 1970s, AI scientists such as Ross Quillian had to demonstrate how to build working language models based on a vocabulary of only 20 words because computer memory of the time was too limited to handle the data structures needed to process more words. Few options were available if a computer couldn’t hold all the data, and a solution was to process key problem information and obtain it from humans who knew it best.
Expert systems were experts not because they based their knowledge on their own learning process, but rather because they collected it from human experts who provided a predigested system of key information taken from studying books, learning from other experts, or discovering it by themselves. It was basically a smart way to externalize knowledge into a machine.
MYCIN: A beginning expert system
An example of one of the first systems of this kind is MYCIN, a system to diagnose blood-clotting diseases or infections caused by bacteria, such as bacteremia (when bacteria infect the blood) and meningitis (inflammation of the membranes that protect the brain and spinal cord). MYCIN recommended the correct dosage of antibiotics by using well over 500 rules, and it relied, when needed, on the doctor using the system. When there wasn’t enough information available — for instance, lab tests were missing — MYCIN then started a consultative dialogue by asking relevant questions to reach a confident diagnosis and therapy.
Written in Lisp as a doctoral dissertation by Edward Shortliffe at Stanford University, MYCIN took more than five years to complete, and it performed better than any junior doctor, reaching the elevated diagnosis accuracy of an experienced doctor. It came from the same laboratory that devised DENDRAL, the first expert system ever created, a few years earlier. DENDRAL, which specializes in organic chemistry, is a challenging application in which brute-force algorithms proved unfeasible when faced with human-based heuristics that rely on field experience.
As for MYCIN’s success, some issues arose:
- First, the terms of responsibility were unclear: If the system were to provide an incorrect diagnosis, who was responsible?
- Second, MYCIN had a usability issue because the doctor had to connect to MYCIN by using a remote terminal to the mainframe in Stanford, a quite difficult and slow process at a time when the internet was still in its infancy. MYCIN still proved its efficacy and usefulness in supporting human decisions, and it paved the way for many other expert systems that proliferated later in the 1970s and 1980s.
The components of expert systems
Generally, expert systems of the time were made of two distinct components: knowledge base and inference engine. The knowledge base retains knowledge as a collection of rules in the form of if-then statements (with if involving one or multiple conditions and then involving conclusion statements). These statements occurred in a symbolic form, differentiating between instances, (single events or facts), classes, and subclasses, which all could be manipulated using Boolean logic or sophisticated first-order logic, which is composed of more possible operations.
First-order logic is a set of operations that goes beyond simply being bound to combine TRUE and FALSE assertions. For instance, it introduces concepts such as FOR ALL and THERE EXISTS, allowing you to deal with statements that may be true but cannot be proved by the evidence you have at hand at that moment.
The inference engine is a set of instructions that tell the system how to manipulate the conditions based on the Boolean logic set of operators such as AND, OR, and NOT. Using this logic set, TRUE or FALSE symbolic conditions can combine into complex reasoning. (When TRUE, a rule is triggered or, technically, “fired”; when FALSE, the rule doesn’t apply.)
Because the system was made at the core of a series of ifs (conditions) and thens (conclusions), and was nested and structured in layers, acquiring initial information helped rule out some conclusions while also helping the system interact with the user concerning information that could lead to an answer. When dealing with the inference engine, common operations by the expert systems were as follows:
- Forward chaining: Available evidence triggered a series of rules and excluded others at each stage. The system initially concentrated on rules that could trigger an end conclusion by firing. This approach is clearly data-driven.
- Backward chaining: The system evaluates every possible conclusion and tries to prove each of them on the basis of the evidence available. This goal-driven approach helps determine which questions to pose and excludes entire sets of goals. MYCIN, described earlier, used backward chaining; progressing from hypothesis backward to …
Expert systems originated in the early days of AI, leveraging human expertise by encoding it into rule-based models. Their logic-driven approach remains foundational in fields such as medical diagnostics, legal reasoning, and chemical analysis. Although modern AI has moved toward data-driven machine learning, expert systems are still used for problems where interpretability and transparency are critical.
Modern insurance companies use AI systems inspired by expert systems to flag suspicious claims for further human review, adapting rules to new fraud patterns as they emerge.
Practitioners often combine rule-based and machine-learning approaches, especially in environments where regulations require clear reasoning trails for decisions.
A computer program that uses a set of rules, often derived from human experts, to make decisions and solve problems in a specific domain.
The part of an expert system that applies logical operations to the knowledge base to reach conclusions or recommendations.
Create a set of “if-then” rules to diagnose whether a plant needs water.
- List at least three conditions (e.g., soil dryness, leaf color, temperature).
- For each condition, write an “if-then” rule (e.g., If soil is dry, then water the plant).
- Share your rule set with a peer and discuss how it could be improved for flexibility.
What are the limitations of encoding human knowledge as a fixed set of rules?
How do expert systems handle situations where information is missing or uncertain?
Imagine you are designing an AI to diagnose medical diseases. How would you decide which rules to include, and how would you handle new diseases that emerge after your system is deployed?
Expert systems learn and adapt by themselves, just like humans do.
Expert systems rely on rules crafted and updated by human experts—they do not autonomously learn new information.
“Expert systems were experts not because they based their knowledge on their own learning process, but rather because they collected it from human experts.”
What is an expert system?
Tap to revealA computer program that makes decisions using rules derived from human expertise in a specific domain.
What is a knowledge base in expert systems?
Tap to revealA collection of rules and facts, often in if-then format, used to guide decision-making.
Define “inference engine”.
Tap to revealThe mechanism in an expert system that applies logical operations to draw conclusions from the knowledge base.
What is backward chaining?
Tap to revealA goal-driven reasoning method where the system tries to prove possible conclusions based on available evidence.
Which component of an expert system is responsible for applying logical rules to reach conclusions?
Expert systems represent a foundational step in AI, using human-derived rules to solve complex, real-world problems—even when information is incomplete or evolving.
The flexibility and reasoning power of expert systems come from their knowledge base and inference engine, enabling dynamic problem-solving beyond simple algorithms.
The Shift
- Traditional algorithms are limited in addressing unpredictable, complex real-world problems.
- Expert systems use human-expert knowledge structured as rules and logical reasoning to bring flexibility and adaptability to AI.
- The evolution of AI began with expert systems, paving the way for more advanced, learning-based approaches.