> For the complete documentation index, see [llms.txt](https://ret2basic.gitbook.io/ctfnote/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ret2basic.gitbook.io/ctfnote/web/prerequisites/information-gathering/server.md).

# Server

Some other common files on the server side include:

* **robots.txt**
  * A `robots.txt` file tells search engine crawlers which pages or files the crawler can or can't request from your site. This is used mainly to avoid overloading your site with requests; it is not a mechanism for keeping a web page out of Google. To keep a web page out of Google, you should use `noindex` directives, or password-protect your page.
* **.htaccess**
  * The `.htaccess` is a distributed configuration file, and is how Apache handles configuration changes on a per-directory basis.
* **.DS\_Store**
  * In the Apple macOS operating system, `.DS_Store` is a file that stores custom attributes of its containing folder, such as the position of icons or the choice of a background image. The name is an abbreviation of Desktop Services Store, reflecting its purpose.
* **readme.md**
  * A README file contains information about other files in a directory or archive of computer software. A form of documentation.
* **Compressed backup files**
  * These might be the names of the website source code backup:
    * `<domain_name>.zip`
    * `<domain_name>.rar`
    * `<domain_name>.tar.gz`