What Is PetalBot?PetalBot is an automatic program of the Petal search engine.

PetalBot is an automatic program of the Petal search engine. The function of PetalBot is to access both PC and mobile websites and establish an index database which enables users to search the content of your site in Petal search engine and present content recommendations for the user in Huawei Assistant and AI Search services, both services are powered by Petal Search enigine. You can identify crawling from Petal by analyzing the User-agent field.

1.2 What Is the User-Agent of PetalBot

Product name

User-Agent

Full User-Agent

PetalBot(PC)

PetalBot

Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot)

PetalBot(mobile)

PetalBot

Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)

1.3 How to Block PetalBot from Visiting Your Site

PetalBot complies with the Internet robots protocol. You can use the robots.txt file to completely prevent PetalBot from accessing your website, or to prevent PetalBot from accessing some files on your website.

Note: Banning PetalBot from accessing your site will make the pages on your site and all search engine services provided by Petal unsearchable in the Petal search engine.

setup recommendations

You can set different crawling rules according to different user-agents of each product, and you can directly prevent the crawling of PetalBot. The following robots can prevent Petal crawling or conditional allow:

User-agent: PetalBot

Disallow: /

User-agent: PetalBot

Allow: /w/api/

Disallow: /trap/

1.4 How Much Pressure Does PetalBot Bring to a Web Server

In order to have a better retrieval results for the target resources, PetalBot needs to maintain a certain amount of crawling of your website. We try not to bring an unreasonable burden on the website, and we will make adjustments based on comprehensive factors such as server capacity, website quality, and website updates. If there is any unreasonable behavior of PetalBot’s visiting, please send your concerns to petalbot@huawei.com.

1.5 How to Judge PetalBot Crawling

You can verify if a web crawler accessing your server really is PetalBot.

To verify PetalBot as the caller:

1. Run a reverse DNS lookup on the accessing IP address from your logs, using the host command.

2. Verify that the domain name is in aspiegel.com.

3. Run a forward DNS lookup on the domain name retrieved in step 1 using the host command on the retrieved domain name. Verify that it is the same as the original accessing IP address from your logs.

Example:

$ host 114.119.128.10

10.128.119.114.in-addr.arpa domain name pointer petalbot-114-119-128-10.petalsearch.com

$ host petalbot-114-119-128-10.petalsearch.com

petalbot-114-119-128-10.petalsearch.com has address 114.119.128.10

1.6 Why My Website Can Still Be Found in Petal When It Has Been Already Added to robots.txt

It takes time to update data in the search engine index database. Although PetalBot has stopped accessing the pages on your site, it may take several months to clear the page index information that has already been established in the Petal search engine database. Besides, please check if your robots configuration is correct.If your rejection is very urgent, please make your request to petalbot@huawei.com.

Shall not be reproduced without permission：ZHLL.COM » What Is PetalBot?PetalBot is an automatic program of the Petal search engine.