# Customize sitemap and robots.txt

Cecil automatically generates `/sitemap.xml` and `/robots.txt`: they are [default pages](/documentation/configuration/#pages-default) rendered with the built-in templates `_default/sitemap.xml.twig` and `_default/robots.txt.twig`.

## How it works

The two pages are declared in the default configuration:

```yaml
pages:
  default:
    robots:
      path: robots
      layout: robots
      output: txt
    sitemap:
      path: sitemap
      layout: sitemap
      output: xml
      changefreq: monthly
      priority: 0.5
```

- the **sitemap** lists all showable pages (published, not virtual, not redirected, not excluded), with their translations;
- **robots.txt** allows everything, disallows redirections and the 404 page, and points to the sitemap.

:::important
Set [`baseurl`](/documentation/configuration/#baseurl) in `cecil.yml`: both files use absolute URLs.
:::

## Exclude a page from the sitemap

Add [`excluded`](/documentation/content/#excluded) to the page front matter:

```yaml
---
title: Thank you
excluded: true
---
```

The page is still published, but hidden from the sitemap and from list pages (home page, sections, etc.). Use `published: false` to not publish it at all.

## Change the default frequency and priority

```yaml
pages:
  default:
    sitemap:
      changefreq: weekly
      priority: 0.8
```

## Add rules to robots.txt

Create a `pages/robots.md` page: its body is appended to the generated rules.

```markdown
---
layout: robots
output: txt
---
User-agent: GPTBot
Disallow: /
```

## Override the templates

Extract the [built-in templates](/documentation/templates/#built-in-templates) to start from the originals:

```bash
php cecil.phar util:templates:extract
```

Or create `layouts/sitemap.xml.twig` (or `layouts/robots.txt.twig`) from scratch. For example, a sitemap that also skips pages with a custom `sitemap_exclude` front matter variable:

```twig
<?xml version="1.0" encoding="utf-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  {%- for p in site.pages.showable|filter(p => not p.sitemap_exclude|default(false))|sort_by_date ~%}
  <url>
    <loc>{{ url(p, {canonical: true}) }}</loc>
    <lastmod>{{ p.date|date('Y-m-d') }}</lastmod>
  </url>
  {%- endfor ~%}
</urlset>
```

:::tip
To disable one of these files, unpublish its default page:

```yaml
pages:
  default:
    sitemap:
      published: false
```

:::