Skip to main content

Block from search engines

Feature RequestDashboard Features

Ability to add a checkbox that will disallow google, bing etc from crawling the website.

Or, add a password protection for HTTP requests.

this is meant for staging&security purposes.

Posted by Aviran Zazon ·
Status: Completed3 comments

Log in to comment and vote

Comments3

  • Arni Johannesson

    Team•

    Jul 23

    This has been released today, found by clicking on the small pencil icon next to your Web Root on the main server info card - only for LAMP/LEMP stacks

  • Arni Johannesson

    Team•

    Apr 28, 2024

    I think this idea has some merit. Enabling password auth for certain paths should be trivial to implement and as for search engine blocking, we could create an "easy" UI for simply writing a robots.txt file containing your directives.

    As Paul mentions however there is potential for collissions here, but with proper information to the user this can be mitigated. We will put this on our backburner :)

  • Epsilon PS _ Paul Schiffer

    •

    Apr 26, 2024

    This could be quite hard to implement because the dashboard would have to check what type of application is running and then pick the right way to "enable" the crawl-block.

    And it would have to make sure to not run into apps that have the same feature, e.g. Wordpress.