AI Assertion

2.1 AI Systems Producing Outputs Without a Descriptive Function

With this account of descriptive functions in hand, we can see that some AI systems produce linguistic outputs that take the characteristic form of assertions, but lack a descriptive function. In their pre-trained form, LLMs are an important example of this phenomenon. LLMs are trained in two stages. First, they are trained by self-supervised learning on large corpuses of text to select the most likely next word (or sub-word token). Second, they can be fine-tuned to improve specific aspects of their performance. Because pre-trained LLMs are trained only to predict the most likely next word, they produce their outputs only because these are among the most likely continuations of the texts with which they are prompted, relative to the statistical models that they embody. This means that their outputs do not have the function of conveying information to a consumer. Pre-trained LLMs include BERT (Devlin et al. 2018), GPT-3 (Brown et al. 2020) and PaLM (Chowdhery et al. 2022).

For example, in the course of responding to a prompt asking it to explain a joke, PaLM produced the following string of words:

• (3)

  A “pod” is … a group of whales. (Chowdhery et al. 2022: 4)

Although this appears to be an assertion, it does not have a descriptive function. This is because PaLM’s training process did not select for outputs which convey information, but instead for outputs which contain likely words. Its training process was not sensitive to whether the outputs it generated in training could be used as a source of information by a consumer. PaLM and other pre-trained LLMs do often produce strings with true conventional meanings, but this is simply because much of the human-generated text on which they are trained consists of true declarative sentences.

It is possible to use LLMs to obtain information or to perform other tasks by careful selection of prompts—that is, of the linguistic inputs which, according to their function, LLMs extend. Researchers in the field describe LLMs as being capable of “few-shot learning,” demonstrated by cases in which including examples of desired outputs in prompts causes them to produce outputs of a similar form. For example, a prompt consisting of two jokes paired with explanations, followed by a third joke, may yield a coherent explanation of the third joke from PaLM (Chowdhery et al. 2022). However, this way of using pre-trained LLMs does not affect the fact that their outputs lack descriptive functions. Examples which demonstrate the task provide context which influences the probabilities of subsequent words, and thus affect LLM outputs. When humans are given demonstrations which help them to understand a task, this can affect their intentions as they attempt to perform it, and hence the functions of their outputs; but there is no reason to think that examples in prompts are processed by LLMs in a similar way. In some cases, it may be helpful to think of human users as giving pre-trained LLMs “expedient functions,” analogous to using a watch as a paperweight or a tree as a sundial, to understand how the intentions behind prompt selection affect LLM functions.⁵

2.2 Measuring Devices

While having a descriptive function is a necessary condition for assertion, it is not sufficient. Accordingly, there are many cases in which AI systems produce outputs which do have descriptive functions, but which still fall short of assertion, as we want to argue in the remainder of this section.

We have already seen an indication of this possibility in the case of the thermometer. Although their outputs have descriptive functions, we can be confident that thermometers do not make assertions about the temperature, because any theory on which they do would be likely to collapse the category of assertion into that of descriptive representation. Some AI systems are closely analogous to thermometers. An example of such a system is the device for classifying skin lesions developed by Esteva et al. (2017).⁶ This device consists of a convolutional neural network trained by supervised learning on a dataset of labelled images of skin lesions. In operation, it takes a photograph of a patch of skin as input, and produces an estimate of the probability of each of a number of types of lesion as output. Its outputs thus contain strings of expressions such as “12% malign melanocytic lesion.” Like a thermometer, this device extracts information from inputs and presents it in the form of one of a pre-determined range of possible outputs, through a process which, once training is complete, is wholly inflexible.

A similar example is the fictional credit-scoring system SmartCredit, discussed by Cappelen and Dever (2021). Cappelen and Dever give a detailed argument for the claim that SmartCredit can produce outputs which predicate the property of having a level of credit risk to an individual, and we agree with the thrust of this argument. But they do not go beyond raising the question of whether such devices can perform assertion.

Despite producing outputs with descriptive functions, these AI systems should not, in our view, be thought of as capable of assertion. Our argument for this point is that these systems are much more like thermometers than like humans in the way that they produce their outputs. These systems are calibrated with reference to a set of training examples, so as to behave in accordance with a certain input-output function, which extracts useful information from the inputs. Their outputs are somewhat more diverse and complex than those of the thermometer, but such differences concern the content of the outputs, not the way in which the outputs are generated.

This means that if we took the view that these systems make assertions, it would be difficult to deny that thermometers do so too. But as we have suggested, stretching the notion of assertion to cover cases like this would make it too broad to be explanatorily interesting or useful.

So, without endorsing a theory of assertion or discussing necessary conditions beyond descriptive function, we take there to be good reasons to deny that the pre-trained LLMs discussed in this section are capable of performing assertions. This fits with a point Cappelen and Dever make about their method of anthropocentric abstraction in theorising about AI behaviour. In discussing the metasemantics of AI outputs, Cappelen and Dever note that

most of our theories of representation are too anthropocentric. They are parochial because they are based on contingent features of our communicative practice. These features are salient to us, but not essential to the nature of content and communication. (Cappelen & Dever 2021: 69)

In order to apply such theories to AI systems, they argue, we have to “abstract away from these contingent and parochial features of human communication to reveal a more abstract pattern that is realizable in many kinds of creatures” (70). However, they also emphasize that anthropocentric abstraction must not be overdone, or else the resulting concepts will become too broad—it is important “to abstract just the right amount” (70). In the current case, abstracting away from human assertion to such an extent that the systems under discussion can be said to be performing assertions would be going too far, as it would collapse the concept of assertion into that of descriptive representation.⁷

To sum up: We have argued that for an output to constitute an assertion it must have a descriptive function: it must have the function of conveying information to an observer or consumer system, so as to cause the consumer to behave as though some condition holds. We have pointed to some AI systems that do not produce outputs with a descriptive function and thus cannot be said to assert. And we have discussed descriptive linguistic outputs of AI systems that nonetheless should not count as assertions. So, a descriptive function is necessary but not sufficient for assertion. In the next section we will consider what else would be required for outputs of AI systems to count as assertions.

3.1 Assertion and Sanctionability

The idea that the practice of assertion is intimately connected with sanctionability goes back at least to Peirce, who writes:

[The act of assertion] would be followed by very real effects, in case the substance of what is asserted should be proved untrue. This ingredient, the assuming of responsibility, which is so prominent in solemn assertion, must be present in every genuine assertion. (1908/1934: 5.546)

While Peirce does not use the term “sanction,” this is clearly what he has in mind in mentioning the “very real effects” of untrue assertions, and he goes on to say that false assertions “endanger the esteem in which the utterer was held [or] entail such real effects as he would avoid” (5.546). He also notes that asserters face “punishment” (5.546) for putting forward falsehoods.

Today, the view that asserting requires sanctionability is contained in the two most widely accepted views of assertion, which focus on normative aspects of assertion: the constitutive norm account and the commitment account. On the constitutive norm account, acts of assertion are governed by a constitutive norm, such as, for example, the norm that one should assert only what one knows (Williamson 2000). Agents who violate this norm by making unwarranted assertions can be held to account for doing so: they can be sanctioned. On commitment accounts, asserting means taking on a responsibility, in the context of social linguistic practice, to justify or defend the content put forward (Peirce 1908/1934; Brandom 1983; 1994). Again, those who take on such responsibility without being able to justify the content put forward can be sanctioned for doing so.

Of course, there are accounts of assertion that do not directly appeal to norms or responsibility. In particular, there are attitude accounts, which claim that assertions express beliefs (Searle 1969; Bach & Harnish 1979), and dynamic accounts, according to which asserting a content means proposing to add that content to the common ground (Stalnaker 1999; 2014). But even these accounts must attend to normative features of assertion to provide a full picture of what it is to assert, and they are often combined with one of the two normative accounts. Stalnaker is explicit about this aspect of his approach:

On the account of the speech act of assertion that I have been using, an assertion is something like a proposal to change the context by adding the content expressed in the assertion to the common ground. I should emphasize that I am not claiming that one can define assertion in terms of a context-change rule, since that rule will govern speech acts that fall under a more generic concept. A full characterization of what an assertion is would also involve norms and commitments. (Stalnaker 2014: 89, original emphasis).

And although Bach’s view is based on the idea that assertions express beliefs, he accepts that a knowledge norm of assertion can be derived from combining his account with a knowledge norm of belief (cf. Bach 2008: 77). So even accounts of assertion that are in the first instance non-normative tend to be spelled out into views that accept assertion as a norm-governed practice. And once assertive commitment or a norm of assertion are in play, so is sanctionability.

Note that if assertion is understood as a norm-governed practice, sanctionability is an essential condition for assertion, not just an incidental feature. Part of what it is to be subject to a norm of assertion is to be sanctionable upon violating it. And part of what it is to take on a justificatory responsibility in asserting is to be sanctionable if one cannot fulfil this responsibility. We thus agree with Mark Jary that sanctionability is a precondition for assertion:

In order for an utterance to have assertoric force, it must also be subject to the cognitive and social safeguards that distinguish assertion. … It is the applicability of these safeguards that distinguishes assertion both from other illocutionary acts and from other forms of information transfer. (Jary 2010: 163–164)

In what follows, we will therefore assume that asserting requires sanctionability.

What, then, does it mean to sanction someone for an unwarranted or insincere assertion? To answer this question, we will begin with a relatively simple possible response, then show that two further elements have to be added to arrive at a plausible result.⁸

The initial approach can be developed from Peirce’s comment about the asserter’s “esteem”: On this view, false or unwarranted assertions are sanctioned by lowering the esteem in which the asserter is held by their interlocutors. In the philosophical debate on assertion, the notion of esteem is most commonly spelled out in terms of trust or credibility. False or unwarranted assertions are sanctioned by relying on the agent’s assertions to a lesser extent in the future, by deducting “credibility points” (Kauppinen 2018: 2). A view of this kind has been defended by Mitch Green:

Assertions, conjectures, suggestions, guesses, presumptions and the like are cousins sharing the property of commitment to a propositional content. … Consequently, a speaker incurs a distinctive vulnerability for each such speech act—including a liability to a loss of credibility and, in some cases, a mandate to defend what she has said if appropriately challenged. These liabilities to error, exposed insincerity, and injunctions to defend put the speaker at risk of losing conversational “weight” in the community in which she has a reputation …. (Green 2009: 157)

Green holds that asserters incur both a liability to a loss of credibility and a mandate to defend the content put forward, but he then ties the latter notion back to the possibility of losing epistemic status. So, on Green’s view, sanctioning assertions boils down to a loss of credibility.

While it seems right that sanctioning asserters involves a reduction in trust, this cannot be all there is to the sanctions in question. After all, we can (and often do) deduct credibility points from agents or systems that clearly cannot assert. For example, if we find out that a thermometer has produced inaccurate outputs then we are less likely to rely on it in the future. But it would be wrong to say that, in doing so, we are sanctioning the thermometer. Just as it is clear that thermometers cannot assert, it is clear that they cannot be sanctioned in the way asserters can be sanctioned. This indicates that sanctioning assertions involves more than just a reduction of trust.

Why doesn’t a reduction in trust amount to a sanction for the thermometer? One obvious reason is that there is no way of communicating this reaction to the thermometer and to thus improve its performance. This indicates that asserters must have some capacity to keep track of the extent to which they are trusted. Once this element is added, sanctioning asserters requires not only reducing trust, but also letting them know about this reduction and thus potentially influencing them, making unwarranted or false assertions less likely in the future. This view of sanctions has been defended by Kauppinen:

[W]hen others reduce their epistemic trust in us and manifest this in their behavior, it is genuinely a way of holding us accountable, a sanction-like response that is (other things being equal) apt to make us change our behavior …. (Kauppinen 2018: 8)

By requiring sanctions to be apt to change the asserter’s behaviour, this approach avoids the problem of having to see simple thermometers as sanctionable. While we can reduce trust in thermometers, they cannot be aware of this and thus it will never change their behaviour.

However, a variation of the thermometer case shows that these first two elements still do not add up to a plausible view of sanctioning. Consider a thermometer that has a feedback button that can be pushed in case of inaccurate measurements. Once the button has been pushed a certain number of times, the thermometer enters an automatic routine of self-cleaning and recalibration. This more sophisticated thermometer fulfils both elements required for sanctioning on Kauppinen’s view, but it still seems implausible to say that the thermometer is producing assertions or that it can be sanctioned for its outputs. Granting these features to the sophisticated thermometer would still be driving anthropocentric abstraction too far. Sanctioning requires a third element.

A good candidate for this third element of sanctioning is the following: (negative) sanctions have to be bad and undesirable for asserters. This is what Peirce (1908/1934: 5.546) is concerned with when he states that asserters face “punishment” (5.546) for putting forward falsehoods. And although Kauppinen argues that blame or punishment is unfitting in the case of assertion (as the normativity in play is non-moral—in his view it is epistemic), he emphasizes that sanctions are undesirable for asserters; they amount to a “kind of analogue of punishment” (Kauppinen 2018: 7). Once this element is added, (negative) sanctions for assertions require (i) that interlocutors keep track of asserters’ credibility, (ii) that asserters are sensitive to this, and (iii) that losing credibility is bad for asserters. This is the view of sanctioning we will work with in what follows.

One point in its favour is that it provides the correct result for the cases of the sophisticated thermometer. While we can adapt our credibility in the thermometer and there is a sense in which it is sensitive to our credibility ratings, there is clearly no sense in which it is bad or undesirable for the thermometer if we reduce our credibility.

According to Kauppinen, elements (ii) and (iii) go together, at least for humans: if we reduce trust in an asserter, that is bad for them. He uses Fricker’s (2007) view on epistemic injustice to argue that “losing credibility—rightly or wrongly—is bad and undesirable for a person, and therefore something that can function as an analogue of the harm involved in punishing” (Kauppinen 2018: 7). But the sophisticated thermometer illustrates that this verdict does not apply to all potential asserters: it can be sanctioned in a light-weight sense that involves only (i) and (ii), but not in a full sense that involves (i), (ii) and (iii).

Taking a step back will help to make clear the importance of condition (iii) in our account of sanctions for assertion. As Peirce recognized, sanctions are built into the practice of assertion; part of what it is to make an assertion is to make oneself liable to be sanctioned. Because sanctions are bad for us, we have reasons to conform to the norms of assertion, and there is substance to the commitments we make when we assert. We are subject to the norms of assertion. Without condition (iii), asserters would only need to be sensitive to loss of credibility, and influenced in their verbal behaviour by this sensitivity. A system can be sensitive in this way without being capable of acting for reasons, making commitments or being subject to social norms, as the sophisticated thermometer shows. Somewhat similarly, LLMs mimic human language use, and are sensitive in many ways to the norms that govern it, without being subject to these norms.⁹

3.2 Can Fine-Tuned LLMs be Sanctioned?

Let us now consider whether fine-tuned LLMs, such as LaMDA and Sparrow (Glaese et al. 2022), are sanctionable. To do so, we will begin by describing their design objectives and training processes.

LaMDA was built by fine-tuning a pre-trained LLM with the aim of improving the quality, safety and groundedness of its outputs (Thoppilan et al. 2022). Groundedness of an output was defined as its being based on a known and established source of information. To fine-tune for quality and safety, the researchers began by defining these objectives, then crowdsourced ratings according to the defined criteria of outputs of the pre-trained model. They then used the data obtained from this process to train discriminators to re-rank or filter out candidate model outputs. To fine-tune for groundedness, they asked crowdworkers to examine model outputs with a set of tools—a calculator, a translator and an information retrieval system with access to the internet—to check and correct any verifiable factual claims. They then trained the model, which had access to the same tools, to mimic this behaviour and correct candidate outputs before producing them.

In fine-tuning for groundedness, Thoppilan et al. aimed to both improve the accuracy of some of the apparent assertions produced by LaMDA, and ensure that the content of these outputs can be traced back to known sources. The reason why this aim applied to only some outputs which appear to be assertions is that it is possible for LaMDA to produce apparent assertions which are unverifiable. For instance, it might output “John baked three cakes last week” in a context in which ‘John’ does not refer to any identifiable person (Thoppilan et al. 2022: 6). Nonetheless, this training did affect the function of many of LaMDA’s outputs. The aim of the LaMDA project was partly to build an LLM which would produce outputs that users could trust, and the fine-tuning process involved selection for features making true outputs more likely. So LaMDA does produce outputs with descriptive functions, such as (2), the output about Mt. Everest which we quoted above.

Sparrow is similar to LaMDA: based on a pre-trained LLM, it is fine-tuned to be helpful, harmless and correct (Glaese et al. 2022). For our purposes, the notable difference between these two systems is that Sparrow is fine-tuned in part by reinforcement learning from human feedback (RLHF).¹⁰ This means that crowdworkers assessed examples of outputs from Sparrow, the resulting data was used to train reward models, and these models provided a reward function which was used to fine-tune Sparrow itself by reinforcement learning. The reward model would give Sparrow positive reward for outputs which were similar to those that humans preferred (when rating according to the three objectives of helpfulness, harmlessness and correctness), and negative reward for outputs of the kind that humans did not prefer.

This difference is notable because it relates to the element (ii) from our discussion on sanctions: the condition that to be capable of assertion, a system must be sensitive to the level of trust which users place in it. Like many current AI systems, LaMDA and Sparrow both have their weights fixed for deployment after an initial training phase, so their dispositions cannot change after this phase. However, it is plausible that sensitivity to trust during the training phase could suffice for assertion even in deployment, because outputs during deployment could be explained partly by the earlier sensitivity, in roughly the way that a human’s choices about what to say are explained by their earlier sensitivity to various social forces. In training, Sparrow was arguably sensitive to trust, and in particular to judgments that its outputs are untrustworthy: when crowdworkers judged that outputs were incorrect, their preference affected the reward model in a way which would ultimately tend to guide Sparrow away from outputs of this kind. Accordingly, raters judged that Sparrow was more trustworthy than the pre-trained model (Glaese et al. 2022). Whether LaMDA meets this condition is more doubtful, because the fine-tuning method used to improve groundedness was based on mimicking the methods which humans used to check and correct claims, rather than on ratings of outputs.

In any case, neither LaMDA nor Sparrow are good candidates for producing assertions, if sanctions for asserting require (i) that interlocutors keep track of asserters’ credibility, (ii) that asserters are sensitive to this, and (iii) that losing credibility is bad for asserters. This is because element (iii) is not satisfied. Even if humans negatively assess their outputs, deduct credibility points and provide according feedback to the systems, there is no sense in which this is bad for the systems. This is because such systems do not have interests of their own, and as a result nothing can be good or bad for them.

Why think that LaMDA and Sparrow have no interests of their own? We do not rule out the possibility that future AI systems could have interests. However, for this to be the case, it would likely be necessary either that the system was conscious or that it cared about some properties of itself or its environment (Kamm 2007; Kagan 2019). It is not likely that either system is conscious because there is little reason to think that they have the necessary cognitive architecture (Dehaene et al. 2017; Butlin, Long et al. 2023). They are similarly unlikely to have the emotions or higher-order preferences which philosophers associate with caring (Shoemaker 2003; Jaworska 2007; Sripada 2015).

Machine learning models are trained to approximate objective functions, so it might be suggested that these systems have an interest in achieving this. In certain respects the relationship between a model and its objective function may be seen as analogous to the relationship between a person and their goals. One such respect is that worse performance with respect to the objective function, during training, typically results in greater changes in subsequent behaviour than better performance. Another is that, partly as a consequence, more training tends to result in better performance.¹¹ But the idea that objective functions describe objectives or goals that AI systems pursue is an oversimplification (Butlin 2022). The objective function is simply an input-output function describing behaviour that the system will come to approximate through training if it functions correctly, and models trained with objective functions can be extremely simple (e.g. decision trees; Russell & Norvig 2010). And in any case, it is questionable whether having goals entails having interests.

So this is our main result: If asserting requires the possibility of being sanctioned, and if sanctions are spelled out in a full sense that includes their undesirability, then even fine-tuned LLMs that are designed to produce accurate outputs cannot perform assertions. This suggests that the optimistic outlook on AI assertion by Kasirzadeh & Gabriel (2023), Freiman & Miller (2020), Evans et al. (2021), and others is mistaken.

We think that there at least two further reasons why this result is important. For one thing, it suggests that verdicts on whether a certain system is capable of assertion or other speech acts cannot be made on the basis of considering the system’s outputs alone. Rather, it is essential to assess whether the system can have desires or interests of its own, and thus to assess its architecture and training as well as its outputs. In this respect, there is a parallel with the case of Blake Lemoine’s (2022) attribution of sentience to LaMDA: Lemoine focuses only on LaMDA’s outputs to argue that the system is sentient, without taking into account features of its architecture, for which he has been widely criticized (Chalmers 2023). And just as we cannot tell solely on the basis of its outputs whether an AI system is sentient, we cannot tell solely on the basis of its outputs whether an AI system can perform assertions.

Secondly, Sparrow and similar fine-tuned LLMs provide real-world counterexamples to the claim that losing credibility (and finding out about it) is always an analogue of punishment. While fine-tuned LLMs can be sensitive to credibility scores and while this can even influence their outputs, they are not sanctionable in a full sense, as low credibility scores are not bad or undesirable for them. This shows that the element of sensitivity to credibility and the element of undesirability of low credibility scores in the account of sanctioning motivated above can come apart, and it raises the question whether this might also happen in the case of human assertion.