Foundations of Generalization

Introduction

Human intelligence and its ability to generalize stand behind the rapid progress of our civilization. They enable us to adapt to new environments and solve previously unknown tasks. However, the mechanisms of generalization remain unclear, which limits our progress in cognitive sciences, especially in artificial intelligence.

In this paper, we introduce comparable properties and their ranges as low-level components of intelligence that enable its rich set of higher-order functions. We describe differences and similarities as operations on ranges of properties.

We continue with the analysis of specialization, which allows us to categorize any object based on its properties. We view this process as a necessary precursor to generalization and a crucial stage for filling the set of defining features of any concept.

Generalization is then considered as the reverse process of forgetting differences between sibling subclasses. We then proceed to show that generalization applies not only to objects but also to actions and other syntactic constituents. We may apply this approach to the analysis of complex concepts and abstract concepts such as functions.

1 Definition of Intelligence

We will propose a working definition of intelligence based on which we will analyze generalization.

Intelligence is the ability to handle differences.

There are many definitions of intelligence, which can be expressed in terms of each other. For example, one of the definitions relies on the achievement of goals. If we view the desired state as different from the current one (as reflected by the properties of the two states) then to achieve the goal, we need to find the course of action that will change respective properties appropriately.

The advantage of the proposed definition is that it provides low-level components, namely comparable properties, of intelligence, its atoms, based on which it is possible to construct the molecules - higher-order functions of intelligence. In this paper, we will show how comparable properties help in explaining generalization.

2 Unique and Interchangeable Objects

To understand generalization, one must understand the difference between unique and interchangeable objects.

The real-world phenomena are characterized by variety and uniqueness. The latter is especially interesting as it applies to objects over time. As Heraclitus put it, "No man ever steps in the same river twice, for it's not the same river and he's not the same man." Events are unique and the experiences of any event by its participants are unique.

But for practical purposes, we learned how to group phenomena so that they are interchangeable. This is true about the exact copies of products manufactured and sold. This is true about completely different physics teachers who still can substitute for each other in a class. These ideas are explored deeper in Deleuze [1] who differentiates between generality and repetitions, "The exchange or substitution of particulars defines our conduct in relation to generality. ... Repetition ... concerns non-exchangeable and non-substitutable singularities. ... To repeat is to behave in a certain manner, but in relation to something unique or singular which has no equal or equivalent."

All objects are unique and thus different from each other. Interchangeable objects have similarities that enable their substitution. Understanding generalization requires an understanding of differences and similarities.

3 Properties and Differences

Differences and similarities rely on comparisons. How do we perform comparisons?

To perform a comparison of two objects we need to first agree on a property shared by both objects based on which we will compare them. We compare objects one property at a time. For example, we may compare trees by height, age, or kind, but not by all those properties at once.

To compare concepts, we also use one property at a time. Ayn Rand [2] stresses the importance of shared properties for comparisons, "All conceptual differentiations are made in terms of commensurable characteristics". Another quote from Ayn Rand sheds light on what concepts are, "Another example of implicit measurement can be seen in the process of forming concepts of colors. Man forms such concepts by observing that the various shades of blue are similar, as against the shades of red, and thus differentiating the range of blue from the range of red, of yellow, etc." In the case of colors, each basic color is a range of possible values. It is the ranges of possible values what concepts are.

If we consider properties as dimensions of the concept space, then ranges of different properties corresponding to the same concept form multidimensional pockets of that space. We may also call concepts multidimensional ranges.

When the property value for a given object falls into a range, the object corresponds to the concept referring to that range. Objects falling outside the range correspond to different concepts. Within any property, ranges depend on the property itself and on the practical purpose at hand. A child may call the mother’s lips “red”, but the mother knows better that they are “Flare Magenta”.

Another factor to consider here is vagueness (see the SEP entry [3]). Within a range, values may be hard to differentiate. Here is how [3] explains what stands behind the range differentiation, "Where there is no perceived need for a decision, criteria are left undeveloped." If necessary, we may introduce more ranges and correct the location of borderlines between them.

Note that ranges may overlap. We may have a classical range, “from 6 AM to 10 AM”, or a range based on allowances, “let’s meet at about 2 PM”.

Mistakes in the evaluation of objects, with respect to which concept they correspond to, happen most often due to the so-called borderline cases. But those mistakes are not equal. For example, it is one thing to say, "Good morning!" when it is slightly past 10 AM and it is another thing when it is slightly past 10 PM.

4 Ranges and Similarities

Both the exemplar theory [4] and prototype theory [5] propose to define a concept via exemplars or prototypes respectively. Then a decision on whether any given object belongs to any concept is made through the comparison of that object to either exemplars or prototypes. If the object is similar to those then the decision is positive and negative otherwise.

However, there is a problem with this approach. This problem is computational. Neither theory states how to select a concept to compare an object to. Do we have to compare an object at hand to all the known concepts? Neither theory states how to select properties based on which to compare similarities. Neither theory states what level of similarity is sufficient to announce a matched concept. Without these details, the volume of computations will be unrealistic for processing in real-time.

It is natural to focus on similarities of properties, but some properties are more important than others. For example, if we have an exemplar (or prototype) of a cat and we want to compare any cat to that exemplar (prototype), do we compare colors? Do we compare sizes and weights? As we see, some concepts may have properties that are not critical for defining those concepts. Some properties are optional - they may or may not be similar.

The above considerations make similarities of properties unreliable for defining concepts. One more argument against the blind use of similarities comes from metaphors. Note that we do not recall anything ordinary or average when describing something to someone who has not seen that. For example, we do not say, "He reminds me of my brother who is also of average height." On the contrary, we use something extraordinary as a reminder.

5 Context

Note that concepts may be relative. When we say, "it was loud", we refer to different sound levels in a library and at an airport. We may use the word "long" for a fishing rod or for a proverbial straw.

It is possible that in any given context we evaluate the available objects in terms of any property and distribute them into different buckets: below average, average, above average. The absolute measurements of objects may be irrelevant for assigning concepts to them, such as "loud" or "long".

6 Specialization vs Generalization

Fodor [6] claims, "I’m going to claim, to put it very roughly, that satisfying the meta-physically necessary conditions for having one concept never requires satisfying the metaphysically necessary conditions for having any other concept." His hypothesis later became known as "atomism". Atomic concepts have all the same problems as exemplars and prototypes.

As an alternative, consider decision trees based on the binary search algorithm. Charles Sanders Peirce [7] analyzing the game 20 Questions wrote this about "good questioners": "Thus twenty skilful hypotheses will ascertain what two hundred thou-sand stupid ones might fail to do." Let us analyze the role of properties in that game and what it can teach us about the mechanism of generalization.

At the start, all the concepts are possible. The first "yes/no" question should apply to all of them and the answer should discard roughly half of them. All the following questions should apply to the remaining concepts and discard roughly half of them. Note that the following questions depend on the previous answers, that is, they depend on semantics. The fact that at each level a question applies to a narrower range of concepts than the previous one and a broader range than the next one is important.

Consider now a variation of that game based on properties and ranges. When we recognize an object, we roughly know the range it falls onto with respect to any of its properties. Consider the property "profession" and its range "teacher". This range in turn is another property, which can be broken down into ranges based on other properties. The property "subject", for example, may narrow down the search to "physics" teacher. Physics has subareas, which narrows down the concept space even further. But a student searching for a private tutor may be more interested in other properties, for example, "language" or "location". Note that the property "height" is almost irrelevant for the selection of a teacher but quite important for the selection of a basketball player. It implies that relevant "breakdown" properties depend on semantics, as was noted previously.

The breakdown process is widely known as specialization. Note the role of differences in this process. We consider all the objects within a class/concept. We take a semantically relevant property and break the class into subclasses based on the ranges of that property. Each subclass is defined by the range of the property and that differ-entiates it from other subclasses. Note that instances of a subclass may have some other properties and similarities (shared ranges) based on them. But their belonging to the class is defined by differences from sibling classes, not by similarities of intra-class instances.

When discussing similarities, we mentioned the difficulty of the exemplar or proto-type theories with sets of properties for comparison when deciding on a concept for a given object. Now we can answer that question. When the specialization process leaves us in a leaf node with the only concept fitting our object, we may go up the search tree and stack all the defining features for that concept. Note that computationally any concept recognition with this approach boils down to roughly 20 simple comparisons - quite affordable for real-time applications.

Note how dictionaries reflect this in definitions. Definitions mention a parent class and a differentiating factor based on some property. For example, the Cambridge Dictionary [8] defines "armchair" as "a comfortable chair with sides that support your arms". Note two differences from just a chair - "comfortable" and "sides that support your arms".

What is generalization? Jorge Luis Borges [9] explains it this way, "To think is to forget a difference, to generalize, to abstract." For simplicity, take classes of "red chairs" and "green chairs". They were introduced based on differences in the "color" property of the "chair" class instances. To generalize back to the "chair" class we only need to forget the differences in color.

Generalization as well as specialization depends on semantics. Depending on which differences we decide to ignore we may generalize in different directions in the concept space.

Concepts are recognized in comparison (to sibling concepts). That is an important piece missing in Fodor's argument for atomic concepts.

7 Multiple Dimensions

Objects have multiple properties. Each of those properties provides a unique point of view on those objects. For example, "a cup of tea" may be viewed as a container with liquid in physics class, as a product in a coffee shop, or even as a process with duration in the phrase "two cups of tea later".

Depending on how we view a given object we may generalize it differently. A person may generalize to a mammal as a biological entity, to a relative as a family member, to an employee in an organization, etc. In each case, we forget different differences and pay attention to different properties.

Objects are carriers of properties. It is important to remember that objects are multi-dimensional but not omnidimensional. The limited number of properties and ranges makes it easier to differentiate objects from each other and out of context. And even among those properties that any object has not all properties are equally relevant for different purposes.

8 Actions Change Properties

Viewed statically objects have properties. Actions change some of those properties. This change is semantic because what properties are affected is determined by the nature of an action.

Natural languages reflect that most actions have subjects and objects. For the purposes of this discussion, we may view actions as dimension-lowering devices. Actions do not treat all the object properties equally. We may say that an action "expects" a subject to have some (limited set of) properties and an object to have some (not necessarily the same) properties. For example, the "eat" action expects an "eater" as a subject and "food" as an object.

Rules are often expressed at a class level. But there are exceptions to any rule. Exceptions are the cases when a rule does not work. In other words, the are class instances with differences in terms of adhering to the rule. Differences introduce subclasses. They do it based on properties not considered previously. What properties have to be considered? It may depend on the semantics of each action. One has to pay attention to the differences in effects.

Actions both have properties of their own and change the properties of objects affected. We know the role of properties in generalization. Does that mean that we may witness two different manifestations of generalization for actions? Indeed, we may. We may love a particular song or music more generally. We may boil an egg or cook it more generally. Ultimate generalization may be represented by auxiliary verbs and pro-nouns, for example, "Do it!"

9 Generalization beyond Objects

Besides subjects, objects, and verbs (standing for actions), sentences have other constituents. Can we expect generalization there? It turns out we can.

Consider the adverbials of time. We may have "In the first half, team A was better, in the second half, team B was better, but overall the game was equal." For some events, there are "before" and "after" periods viewed differently by some people. But there are other people who ignore those differences, ignoring "before" and "after" as well as the events.

The adverbials of location are generalizable as well. "Do you want me to clean up on my table? No, clean up the whole room." Note that subtle syntactic changes from adverbials to objects do not change the semantic role of the location where an action of cleaning has to be performed.

10 Complex Concepts

There are primitive objects like a ball or complex ones like a snooker table. The same applies to actions - "drop a brick" vs "erect a building". We may consider components and their composition as properties of complex concepts and use the above principles to handle their recognition and generalization.

Note that complex concepts also have defining features or components, which in the case of complex concepts may be considered obligatory components. For example, we may imagine a table without legs but not without a surface. Optional may be not only components but also their properties, for example, the number and position of table legs may vary.

Even though we also may view complex concepts through the prism of multiple properties, usually complex and especially man-made concepts and objects that embody them serve a very specific purpose. We may call it the primary purpose. Proverbially we can hit nails with a microscope but its primary function is still to look at tiny objects.

We may generalize concepts based on their function be it primary or not. For example, airplanes and dirigibles all generalize to flying machines even though they have little in common in terms of physical properties. If we consider "function" as some kind of abstract property it will allow us to handle it in the way we already described. The only thing left is to find differences between different functions.

Conclusions

We started this paper by defining intelligence as the differentiating ability. Using that as a working definition, we considered comparable properties and their ranges and their role in determining differences and similarities of concepts. We proposed an alternative to Fodor's atomic concepts, namely that concepts are recognized through comparisons, which determine their defining features. This creates a computationally feasible approach to categorization, specialization, and generalization. We considered how this approach may apply to actions and other syntactic constituents. We showed that even complex concepts can be handled via this approach.

Not only did we show how differences figure in determining concepts, but we explained the mechanism in which comparable properties and their ranges enable specialization and generalization - two important mechanisms of intelligence.

References

1. Deleuze, G.: Difference and repetition. London: Athlone Press (1994).

2. Rand, A.: Introduction to objectivist epistemology. New York, N.Y.: New American Library. Edited by Leonard Peikoff & Harry Binswanger (1966).

3. Sorensen, R.: Vagueness. Stanford Encyclopedia of Philosophy (2006). http://plato.stan-ford.edu/entries/vagueness/#3

4. Medin, D. L., Schaffer, M. M.: Context theory of classification learning. Psychological Review, 85, 207–238 (1978). doi:10.1037/0033-295X.85.3.207

5. Rosch, E. H.: Natural categories. Cognitive Psychology. 4 (3): 328–350 (1973). doi:10.1016/0010-0285(73)90017-0. ISSN 0010-0285.

6. Fodor, J. A.: Concepts: Where Cognitive Science Went Wrong. Oxford, GB: Oxford University Press, (1998).

7. Peirce, C. S.: On The Logic of Drawing History from Ancient Documents, Especially from Testimonies. manuscript corresponding to an abstract delivered at the National Academy of Sciences meeting of November 1901. Published in 1958 in Collected Papers v. 7, paragraphs 162–231; see 220. Reprinted (first half) in 1998 in The Essential Peirce v. 2, pp. 75–114 (1901).

8. Cambridge Dictionary page https://dictionary.cambridge.org/dictionary/english/armchair

9. Borges, J. L.: Funes, the Memorious. The Argentina Reader: History, Culture, Politics, edited by Gabriela Nouzeilles, Graciela Montaldo, Robin Kirk and Orin Starn, New York, USA: Duke University Press, (2002), pp. 306-312. https://doi.org/10.1515/9780822384182-045