A robot does not need to solve every intellectual problem to be useful. It does, however, need to cope with ordinary situations that formal instructions leave out: a chair blocking a route, a cup that may be full, a drawer that sticks, a person on a private call, or an object that looks disposable but is valuable.
That is the challenge of robot common sense. In robotics, it is the practical ability to interpret a situation using ordinary knowledge, anticipate likely consequences and choose behavior that fits physical and social context. It is not a single feature, a database of facts or a synonym for general intelligence. It is a collection of capabilities linking perception, action, prediction, uncertainty and judgment.
Humans acquire much of this knowledge gradually through experience, observation, language, correction and life around other people. Robots need enough of it to remain useful when a room is cluttered, an object is unfamiliar, a goal is underspecified or a person changes the situation halfway through.
The central difficulty is straightforward: seeing the world is not the same as understanding what to do in it.
Why robot common sense is harder than it sounds
Consider a robot asked to clear a table. Identifying a mug, plate and bottle is useful, but it is only a first step. The robot may also need to determine whether the mug is hot, empty or fragile; whether the bottle has a loose cap; whether the plate is balanced on another object; and where the items should go without making a mess or interrupting someone nearby.
A robot can label an object correctly and still make a poor decision. Object recognition answers a question such as, “What does this look like?” Common-sense reasoning must address harder questions: “What can I do with it?”, “What could go wrong?”, “What does this person expect?” and “What should I do if I am not sure?”
This gap helps explain why a successful controlled demonstration does not automatically establish dependable performance in homes, hospitals or public spaces. Demonstrations can limit object types, lighting, surfaces, routes and possible outcomes. Real environments contain clutter, worn materials, improvised arrangements, hidden hazards and people whose preferences cannot be inferred from a task label alone.
Robotics and common sense therefore involve more than improving image classification or command following. The challenge is making different forms of knowledge work together while the machine moves through a changing world.
The hidden knowledge humans learn from everyday life
People learn a great deal about ordinary objects without formal instruction. A container can hold something, but it can also leak or spill. A towel can dry a surface, cushion a fragile item or conceal what lies beneath it. A door may open toward or away from a person. A wet floor changes how safely someone can walk.
Much of this knowledge is tacit. Someone carrying a full bowl across a kitchen may adjust their grip, avoid sudden turns, notice obstacles and choose a route without listing every calculation involved. If another person is working at the counter, they may wait rather than squeeze past. These decisions are embodied and context-dependent.
For a robot, such adjustments must be represented through learned associations, physical models, explicit rules, accumulated memories or a combination of methods. The system must connect what its sensors detect with what its body can do and what its actions might cause.
This is why physical intelligence cannot be reduced to verbal knowledge. A system may be able to state that glass is fragile while lacking the perception, grip control and contact sensitivity needed to handle an actual glass safely. Conversely, a robot may execute a stable grasp without understanding that it should not move an object while someone is using it.
Affordances: what objects invite or allow
A useful concept for understanding robot common sense is the affordance. In ecological psychology, affordances describe action possibilities offered by an environment relative to an organism. A step may afford climbing for an adult but not for a small child. A handle may afford pulling only if it can be reached and gripped. A chair may afford sitting, but not if it is broken, occupied or being used as a step stool.
For robots, robot affordances are similarly relational. A cardboard box might be liftable by one robot and too heavy for another. It may be easy to push on a smooth floor but not on carpet. A cabinet handle may be visible yet inaccessible because another object blocks the robot’s arm. The relevant question is not simply what an object is, but what action it supports for this machine in this setting.
Estimating affordances can require several forms of information:
- Visual perception: shape, orientation, surface features, likely openings and possible grasp points.
- Touch and force sensing: slipperiness, compliance, contact pressure, weight distribution and unexpected resistance.
- Exploratory movement: small motions can reveal whether an item is attached, unstable or heavier than it appears.
- Learned experience: previous interactions can help a robot generalize from similar objects and materials.
- Knowledge of its own body: reach, grip strength, balance, available tools and movement limits.
Affordances are not fixed labels attached to objects. A lid can cover a container, but it may also be lifted, turned or set aside. Whether any of those actions is appropriate depends on the goal and likely consequences.
Consequences: planning beyond the next movement
Physical competence requires more than finding an action that works immediately. A robot also needs to consider what happens next. Pulling a cord may unplug a device. Opening a cupboard may block a passage. Moving a package may expose personal information on its label. Placing a tray at the edge of a counter may complete an immediate task while creating a later risk of falling.
Robot planning involves choosing action sequences that move toward a goal while accounting for constraints, side effects and changing conditions. A reactive system may avoid a collision in the moment, yet be less able to account for longer-term effects, such as leaving an obstacle in a hallway or interfering with another person’s task.
Robotics research uses several approaches to anticipate outcomes. Some systems use geometric or physical models. Others learn predictive models from data, estimating how objects may move when pushed, grasped or placed. Simulation can expose a robot to many variations before physical trials, but simulated materials, contacts and disorder do not perfectly match the real world. Cameras, force sensors and joint sensors remain important when reality differs from expectation.
The sensible action is often deliberately modest. A robot may nudge an object before lifting it, choose a route with more clearance, place an item in a stable temporary location or pause for clarification. In high-consequence contexts, cautious action can be appropriate reasoning under uncertainty.
Ordinary objects are not ordinary to a machine
Human environments are full of variation disguised as familiarity. A cup may be ceramic, paper, insulated metal, cracked glass or a child’s spill-proof container. A bin may be for recycling, waste, confidential documents or donation. A button may control an appliance, summon help or be decorative. The same item can have different functions depending on its location and what people are doing around it.
Laboratory tasks often reduce this ambiguity. Objects may be clean, separated, consistently positioned and selected from a known set. That can make progress easier to measure, but it does not remove the challenge of partially hidden items, handwritten labels, open containers, tangled cables, reflective surfaces and objects altered by wear or repair.
Robot perception must therefore extend beyond naming visible categories. A capable system needs to track relationships: what is inside what, what is supporting what, which object is near an edge, which tool belongs with which station and whether someone is interacting with an item. It may also need history. A surface that was dry a moment ago may now be wet; a door that was open may now be closing.
Recognition alone does not establish weight, friction, contents, ownership or consequence. Those properties must be inferred, sensed, tested or communicated by a person.
Social norms are part of physical intelligence
Robots working around people must navigate unwritten rules as well as walls and furniture. A machine moving through a hallway should avoid physical contact, but it may also need to yield in a narrow space, give room to someone carrying an item and avoid approaching quickly from behind.
Social expectations also shape manipulation. A robot should be cautious about reaching across someone’s body, handling sensitive paperwork, moving medication or entering private spaces. If a person appears to be struggling with a dropped item, an offer of help may be suitable; abruptly grabbing it may not be.
Socially aware robots face an important distinction: predicting what people usually do is not the same as behaving in a way people find acceptable. Appropriate behavior may involve consent, privacy, clear signaling and respect for human priority.
There is no universal social rulebook that a robot can simply download. Expectations vary across households, workplaces, cultures and situations. Adaptation matters, but it should be bounded. A robot should not infer personal preferences more aggressively than the setting justifies.
How researchers try to build common sense into robots
No single method supplies every ingredient of common-sense reasoning in AI. Robotics research instead combines approaches with different strengths and limitations.
- Demonstrations and imitation learning allow robots to learn from examples of human behavior. They can be effective when examples cover the task, but may fail when a new situation differs in important ways.
- Reinforcement learning lets systems improve through trial and feedback. It can discover useful control strategies, but real-world trial and error can be slow, unsafe or impractical around people and fragile equipment.
- Simulation can generate varied training scenarios, including uncommon conditions. Its limitation is the “reality gap” between simulated and physical environments.
- Tactile sensing can provide information cameras may not reliably capture, including slip, pressure, softness and resistance.
- Model-based methods use constraints or predictive models to plan and check actions. Hand-built models can be incomplete, while learned models can be wrong.
- Continual learning aims to help robots improve from new environments and feedback, but adaptation must not degrade previously reliable behavior.
Embodied AI systems increasingly connect vision, language and action. A person may ask a robot to put away groceries, and the system may use images and language to identify likely items and propose steps. Such approaches can make robots easier to instruct, but a language-based plan is not a guarantee of physical success.
A request such as “put the fragile things away” leaves open important questions: which objects are fragile, where “away” is, whether a shelf is stable, whether an item belongs to someone else and whether a container is already full. Language can help define a goal, but it must be grounded in perception, contact, physical checks and safety constraints.
Practical systems may combine learned models with collision limits, task planners, force thresholds, restricted operating zones, safety monitors and human approval for consequential actions. These hybrid designs reflect the fact that reliability often depends on layered safeguards.
The importance of uncertainty and knowing when to ask
A robot that always produces an answer or action can appear decisive. In an uncertain physical world, that confidence can create risk. A more useful system needs ways to represent ambiguity: it may be unsure whether a container holds cleaning fluid or drinking water, whether an object is valuable, whether a doorway leads to a private room or which of two instructions has priority.
Good clarification behavior is selective. Asking about every small detail would make a robot burdensome. Acting freely whenever information is incomplete would make it unreliable. The key question is whether the likely cost of being wrong exceeds the cost of interruption.
Appropriate questions might include:
- “Is this unlabeled container safe to move?”
- “Would you like me to place this document in the cabinet or leave it here?”
- “This object appears delicate. Should I handle it?”
- “The requested route passes through a private area. Do I have permission to enter?”
Uncertainty can also guide behavior without requiring a question. A robot can reduce speed, keep greater distance, choose a reversible movement, avoid a risky grasp or stop before contact. Calibrated hesitation is a capability, not a failure.
Testing robot common sense in the real world
Testing common sense requires more than recording whether a robot completed a task in a familiar setup. Evaluation should examine performance in new homes or workspaces, changing layouts, imperfect objects, unfamiliar users, ambiguous instructions and multi-step tasks.
Useful measures go beyond completion rate. They can include damage, near misses, recovery from errors, clarity about limitations, interruption burden and socially uncomfortable behavior. Testing should also examine noisy sensors, partially obscured objects, incorrect assumptions and unexpected human behavior.
Long-term testing matters because some failures appear only through repeated interaction. A robot may gradually misplace items, adopt an unhelpful shortcut, become unreliable after an environmental change or create friction through persistent small social mistakes.
Safety claims should remain specific to the operating domain. Evidence from a restricted warehouse does not establish safety in a busy hospital corridor or private home. Human-facing systems need defined operating boundaries, accessible stop mechanisms and meaningful ways for people to interrupt or override behavior.
What teaching robots common sense would change
Better common sense could make robots more useful where conditions cannot be standardized. Domestic assistants could handle more variation in kitchens and living spaces. Logistics robots could manage mixed packages and shared routes more effectively. Manufacturing systems could adapt when materials or work cells change. In healthcare, education and elder care, physical and social judgment may matter as much as mechanical capability.
Greater adaptability also expands autonomy and the consequences of mistakes. A robot that can infer goals and act in a wider range of situations has more opportunities to misunderstand a preference, mishandle a personal item or make an inappropriate decision. Common sense should not be treated as permission to remove people from the loop.
Responsible deployment requires choices that make errors recoverable and accountability possible. These can include limited authority over high-consequence actions, clear user controls, logs that support review, transparent task status, conservative defaults and escalation paths for uncertainty. The appropriate balance varies by application, but the principle is consistent: the more a robot’s actions affect people, the more carefully its autonomy should be bounded.
The deeper lesson: intelligence is situated
Common sense is not a hidden encyclopedia waiting to be uploaded into a machine. It is an ongoing relationship among a body, an environment, a goal, other people and the consequences of action. A robot learns that an object is graspable not as an abstract fact alone, but through its own reach, sensors, tools and experience. It learns that an action is acceptable not only because it is physically possible, but because it fits the situation and respects the people in it.
Progress in embodied AI depends on connecting perception, manipulation, prediction, social understanding and humility about uncertainty. Better language models may help robots interpret requests. Better sensors may help them detect contact and material properties. Better planning may help them avoid foreseeable mistakes. None of these components alone is robot common sense.
The durable goal is not a machine that knows everything. It is a machine that can act appropriately when the world is messy, instructions are incomplete and the obvious next step may not be the right one.