“Even with a hypothetically good training data set containing a large number of correctly checked patents, AI algorithms perform much more difficult tasks than other applications. [because] Determining essentiality is subjective. ”
Artificial Intelligence (AI) is delivering tremendous productivity and value improvements in many applications. But AI is not a panacea and is not developed enough to be accurate and reliable everywhere. For example, we need much better AI training data to reliably estimate patent essentiality for standards like 4G and 5G. AI has been advocated by various experts, and he has already been adopted by one patent pool. The reasoning also has a lot of room for improvement.
Most standard essential patents (SEPs) are broadly licensed under comparable licensing benchmarks, but with little or no percentage of patents identified as essential that are actually considered essential). It may be used by courts and patent pools to allocate royalties. The mandatory rate varies by patent holder. declare too much Their patents are perhaps more essential than others. So some licensors (especially the leader his Ericsson, Nokia, InterDigital, Qualcomm) rely heavily on his SEP license income to fund their R&D, so they do such checks accurately and reliably. is important.
magic and reasoning
AI results can be inferred from commonalities such as semantic matches that have no technical or legal basis in patent law. Courts can challenge and explain expert judgments of essentiality, validity, and infringement based on the technical requirements of patent claims and standards. “Because the computer says so” doesn’t wash out there. AI algorithms do not provide reasons for positive and negative decisions in those terms.
A decade ago, with the introduction of voice assistants such as Apple’s Siri and Amazon’s Alexa, there was a big surprise. Speech recognition algorithms were usually able to convert speech to text accurately. AI has even overcome difficulties in dealing with accents and noisy environments. Command responses, however, remain fairly straightforward, such as music selection or standard weather regurgitation. “They were all stupid as rocks,” said Microsoft’s Satya Nadella. financial times last month.
Introducing Chatbots ChatGPT has taken text interpretation to a much higher level. ChatGPT can understand complex instructions and provide sophisticated responses, such as an essay good enough to pass a college exam. Chatbots started by responding to typed queries, but the same functionality can be applied to text generated by speech recognition. However, the main difference between ChatGPT and other AI approaches, which has not been as successful or received much attention recently, is the fine-tuning of the language model with human intervention. This allows an expert to manually rank the generated outputs as feedback for improving his AI model. Thus, AI output is tailored to the subjective preferences of human labelers to whom guidelines are provided, but specific decisions are left to the individual. This Reinforcement Learning from Human Feedback (RLHF) is currently considered very powerful in improving AI reliability.
Unfortunately, AI algorithms can and often do go wrong. For example, longlisting and shortlisting from thousands of CVs in staff recruitment can put you at an unacceptable disadvantage. According to OpenAI, “These models can also produce outputs that reflect untrue, toxic, or harmful emotions. It’s because they’re trained to predict the next word on large datasets of internet texts, rather than running them.In other words, these models aren’t matching users.”
The European Commission’s 2020 pilot study report on the essentiality assessment of SEPs concludes that “automated approaches will not be able to replace human efforts in the short or medium term”.
Garbage In, Garbage Out
AI models, at best, are limited by the quality and quantity of data used for training. Many AI applications can easily take advantage of large amounts of accurate training data. For example, there are many pictures of cats and dogs, and humans can accurately distinguish between them. Based on physical equations and mathematical models, an in-flight aircraft “digital twin” prediction can accurately measure position, altitude, velocity, acceleration, temperature, and airframe distortion and continuously recalibrate.
Other future AI applications will be much more difficult and flawed if training data is poor, inaccurate, or unclear in meaning. This is a significant drawback when using AI to determine and count patents that are considered standard essential.
This AI training data should contain many accurate decisions, such as which patents were determined to be essential and which patents were determined to be non-essential. No such dataset exists. The EC’s pilot study was based largely on a non-random selection of 100 or so exhaustively checked patents, some using claim charts, most of which were found to be essential. I was.
OpenRAN’s Alium patent pool for 4G and 5G contains thousands of patents in its AI training data, which were classified with only cursory manual checks.
inaccuracy and bias
My research shows that manual essentiality checks are imprecise, and results vary widely between studies of essentiality checks. For example, various estimates of the overall mandatory rate including all declarations vary from 50% to 8% for 4G and 5G. The mandatory rate rankings by patent holders are also highly inconsistent. Evaluators often disagree with each other on essentiality determinations, even after thorough checks.
Courts have ruled only on essentiality, infringement, and validity issues for a small percentage and number of declared patents. Perhaps the best decisions out of a good deal are those made for patent pools. But they are not a representative sample of declared patents. The selection is heavily biased to include patents found to be essential. The patent owner usually incurs an evaluation fee of €5,000 to €10,000 for each patent, so he tends to only file declared patents that are relatively likely to be found essential. .
Maximizing the accuracy of manual determination requires the use of claim charts and exhaustive analysis that can take days per patent. This is prohibitively expensive in many cases over a relatively small percentage of declared patents. A cursory check on a large number of patents (usually he takes less than 30 minutes per patent) reduces accuracy and introduces significant system bias in favor of over-declaration. My research also shows that the less precise individual decisions are, the more systematically biased the results. Confidence intervals for the results are relatively wide, and there is considerable sampling error, especially if the essentiality rate is low.
If the checks are inaccurate and the training data is biased, the results of AI-based decisions suffer from the same drawbacks. At least the random sampling error introduced by checking a small number of patents to examine each patent more thoroughly and accurately is unbiased.
Inference from Subjective Judgment
Even with a virtually superior training data set containing a large number of accurately checked patents, AI algorithms perform much more difficult tasks than other applications. Determining essentiality is subjective. Even competent human experts doing exhaustive work often disagree on their decisions regarding the same patent. Technical and legal interpretations of language may differ, words may have different meanings in different contexts, and definitions and uses of language may change over the years. In contrast, the definitions of position, altitude, velocity, acceleration, temperature, and airframe distortion remain unchanged for consistent and accurate measurements.
Limitations on AI inference only make things worse. AI techniques such as semantic text matching provide inferior results to human expert review. Semantic similarity and essentiality are not equivalent. Fine-tuning AI results with the RLHF method can have a significant impact on the performance of AI-based essentiality rating systems, but it also injects more subjective human judgment into the system.
Check confirmation
The accuracy of results using sampling and AI relies on extensive and detailed manual checks based on the sample base and AI training. The optimal combination of detailed manual checks, sampling, and AI techniques is an empirical matter. When essentiality checks and patent counts are used to determine royalties, it is important that methodologies involving detailed evaluation, sampling, and AI inference of various quantities demonstrate their accuracy and cost-effectiveness.
AI tools can help improve rater productivity. For example, when reviewing technical specifications for potential essentiality or prior art (technical specifications published prior to the declared priority date of a patent). However, it cannot replace human decision-making.
Competing licensing platforms, such as independent essentiality assessors and patent pools, must keep the technology they need to make decisions at their disposal. Market forces determine its value and encourage improvement.
However, AI-based essentiality checks should not be enforced by government mandate or involuntarily. There is no transparency and responsibility that justice demands and that licensing parties are entitled to.
Image Source: Deposit Photo
Image ID: 185517136
Author: Von Ramai
