DeepSea Technologies has published a new piece of research outlining a pioneering way of verifying the accuracy – and therefore utility – of a ship’s AI-generated model in real-world conditions.
The new approach was developed by seven of DeepSea’s 13-strong team of research scientists headed up by Dr. Antonis Nikitakis, and presented at the 2022 HullPic Conference in Tullamore, Ireland, in May 2022.
Because the reality of ship-at-sea data is highly variable, most model accuracy figures reported in publications and marketing materials fail to bear relation to the actual utility of those models in real use cases.
DeepSea has researched approaches to solving the technical challenge of boosting models’ ability to understand unseen (“out-of-domain”) conditions for years. However, until today, there has been no benchmark for evaluating this sort of competence within a vessel model. This is a crucial step of AI modelling because the more accurate the virtual model, the more efficient a ship can be made, and vice-versa.
The few models that currently provide an estimation of their accuracy all do so based on testing with data obtained from the same distribution (i.e., representative of similar conditions and containing similar biases) as the data used to train the model. For example, if the model is trained on data from the vessel’s historical behaviour, in a narrow range of well-experienced wind speeds or drafts, it is also tested on data with these speeds and drafts. Thus, the tests performed cannot tell if the model is reproducing the biases in the training data – and whether it will work as well in different, never-seen-before conditions.
Following the publication of this body of work on proving the real-world utility of AI approaches, DeepSea will focus on advocating for common and transparent standards for AI real-world utility in the industry.
Commenting on the research, Dr Nikitakis, DeepSea’s AI Research Director, said:
“This research is an important step in helping our customers and the wider shipping community to understand the true power of an AI-based approach while alleviating its limitations. Coupled with the daily real-world impact we’re seeing on fuel consumption and CII ratings, we believe this sort of information is key to popularising this incredible technology throughout the industry.
“As serious researchers working to popularise the AI approach continue to pursue rigorous methods of proving the real value of what they’re creating, we can also expect end-users looking to employ this sort of technology – in shipping and other sectors – to increasingly demand proof of its utility.”
Dr. Konstantinos Kyriakopoulos, CTO and co-founder of DeepSea, said:
“We designed our AI framework for the direct benefit of the consumer – shipowners, cargo owners, and all other members of the industry who cover fuel costs. I am excited by how, through this research, we can fight the hype and support our honest approach to AI with such compelling evidence. This is exactly what DeepSea was founded to do and what, daily, makes an increasing impact on our clients’ bottom-lines and the sustainable future of the planet.”
DeepSea has publicly released the details of its approach in the hopes that global researchers can utilise it themselves and catalyse greater transparency and common standards across the industry.
