Home

Policy-Based IPsec VPN on FortiGate: Configuration Guide, Limitations, and Modern Best Practices

.

Policy-based IPsec VPN represents a legacy implementation method where traffic selection and encryption decisions are made through firewall security policies rather than routing tables. Unlike route-based VPNs that create virtual tunnel interfaces treated as standard network interfaces, policy-based configurations bind IPsec tunnels directly to firewall policies using the "IPsec" action parameter.

This approach—sometimes called "tunnel-mode VPN"—requires defining both Phase 1 (IKE negotiation) and Phase 2 (IPsec SA parameters) configurations, then associating them with explicit firewall policies that specify source/destination addresses and the tunnel to use for encryption.

Critical Configuration Requirements

Before implementing policy-based IPsec VPN on FortiGate, administrators must verify three essential prerequisites:

  1. NGFW Mode Compatibility: Policy-based IPsec VPN is only available in Profile-based NGFW mode. When FortiGate operates in Policy-based NGFW mode, the "Policy based IPsec VPN" option disappears from Feature Visibility settings entirely, forcing administrators to use route-based configurations exclusively.

  2. Feature Visibility Enablement: Administrators must explicitly enable policy-based IPsec support through:

    config system settings     set gui-policy-based-ipsec enable end

    This setting appears under System → Feature Visibility in the GUI.

  3. Interface Matching Constraint: The destination interface specified in the firewall policy must match the source interface defined in the Phase 1 configuration. Mismatched interfaces represent the most common cause of tunnels failing to appear in the active tunnel list.

Configuration Workflow: Step-by-Step Implementation

Phase 1: Tunnel Establishment Parameters

Phase 1 establishes the IKE security association between endpoints. Critical parameters include:

config vpn ipsec phase1     edit "site-to-site-tunnel"         set interface "wan1"          # Public-facing interface         set peertype any              # Accept connections from any peer type         set proposal aes256-sha256    # Encryption/authentication algorithms         set remote-gw 203.0.113.45    # Remote peer's public IP address         set psksecret ENC             # Pre-shared key (encrypted format)     next end

Phase 2: Traffic Selector Definition

Phase 2 defines the traffic selectors (local/remote subnets) that will traverse the encrypted tunnel. Critical limitation: FortiGate does not support address groups as selectors in policy-based configurations when interoperating with third-party devices. Each unique subnet pair requires its own Phase 2 configuration—creating combinatorial scaling challenges (e.g., 2 local subnets × 10 remote subnets = 20 Phase 2 policies).

config vpn ipsec phase2     edit "site-to-site-tunnel"         set phase1name "site-to-site-tunnel"         set proposal aes256-sha256         # Note: Local/remote selectors default to 0.0.0.0/0 in policy-based mode     next end

Firewall Policy with IPsec Action

The defining characteristic of policy-based VPN: security policies explicitly reference the tunnel using the "IPsec" action rather than standard "ACCEPT":

config firewall policy     edit 1         set srcintf "internal"        # LAN interface         set dstintf "wan1"            # WAN interface matching Phase 1         set srcaddr "192.168.1.0/24"  # Local subnet         set dstaddr "10.0.0.0/24"     # Remote subnet         set action ipsec              # CRITICAL: Not "accept"         set vpntunnel "site-to-site-tunnel"         set inbound enable            # Allow remote-initiated connections         set schedule "always"         set service "ALL"     next end

Important Note: As of FortiOS 7.x, GUI configuration for policy-based IPsec is deprecated. While older documentation shows unchecking "Enable IPsec Interface Mode" in the IPsec Wizard, current versions require CLI configuration exclusively.

Policy-Based vs. Route-Based IPsec: Critical Differences

Feature Policy-Based VPN Route-Based VPN
Traffic Selection Firewall policies with "IPsec" action Routing tables + standard ACCEPT policies
Interface Treatment No virtual tunnel interface created Creates phase1-interface treated as standard interface
Bidirectional Traffic Single policy handles both directions Requires separate policies per direction
NAT/Transparent Mode Supports both modes NAT mode only
L2TP-over-IPsec Supported Not supported
GRE-over-IPsec Not supported Supported
SSL VPN Integration Incompatible Fully compatible
Scalability Poor (combinatorial Phase 2 requirements) Excellent (single Phase 2 with 0.0.0.0/0 selectors)
Third-Party Interop Possible but complex Preferred method; fully compatible with remote policy-based peers

Critical Insight: The remote peer's implementation (policy-based vs. route-based) is irrelevant to tunnel negotiation. IPsec peers never detect the local implementation method—only Phase 1/Phase 2 parameters must match. Administrators can safely deploy route-based VPN locally while connecting to third-party policy-based implementations.

Limitations and Constraints

Policy-based IPsec VPN carries significant operational constraints that make it unsuitable for most modern deployments:

  1. SSL VPN Incompatibility: Policy-based tunnels cannot integrate with SSL VPN interfaces. Organizations requiring both technologies must implement route-based IPsec.

  2. Address Group Limitations: FortiGate cannot extrapolate address groups into multiple security associations required for third-party interoperability. Each subnet pair demands explicit Phase 2 configuration.

  3. NGFW Mode Dependency: Policy-based IPsec becomes unavailable when FortiGate operates in Policy-based NGFW mode—a common enterprise configuration for advanced security features.

  4. Platform Restrictions: High-end platforms (FortiGate 6000F/7000F series) explicitly exclude policy-based IPsec support in their architecture documentation.

  5. Spoke-to-Spoke Limitations: Hub-and-spoke topologies with policy-based VPN cannot natively support direct spoke-to-spoke communication without complex policy configurations. Route-based implementations handle this scenario elegantly through routing protocols.

  6. Troubleshooting Complexity: Debugging requires specialized commands since traffic doesn't traverse standard interface paths:

    diagnose debug flow filter addr 192.168.1.10 diagnose debug flow trace start 100 diagnose debug enable

When Policy-Based Remains Necessary

Despite limitations, policy-based IPsec retains niche applicability:

  • Legacy Third-Party Requirements: When connecting to devices that mandate specific traffic selectors incompatible with route-based implementations (rare in modern deployments)
  • Identical Subnet Scenarios: Bridge-type client-to-site tunnels requiring identical subnets on both sides (e.g., legacy applications demanding flat Layer 2 connectivity)—though this architecture should be avoided where possible
  • L2TP-over-IPsec Requirements: Organizations needing L2TP tunneling over IPsec must use policy-based implementation since route-based lacks this capability

Best Practices and Recommendations

  1. Prefer Route-Based Implementation: Fortinet's official guidance recommends route-based VPN for "greater flexibility and easier configuration." Modern deployments should default to route-based unless specific constraints mandate policy-based.

  2. Use 0.0.0.0/0 Selectors with Route-Based: For maximum interoperability, configure route-based VPN with 0.0.0.0/0 Phase 2 selectors, then control traffic flow through firewall policies and routing—avoiding combinatorial Phase 2 explosion.

    Verify Interface Alignment: Always confirm that the firewall policy's destination interface matches the Phase 1 source interface—this mismatch causes 80% of policy-based tunnel failures.

  3. Enable Bidirectional Initiation: For hub-and-spoke topologies, enable "Allow traffic to be initiated from the remote site" on concentrator policies to support dynamic spoke connections.

  4. Implement IPsec Concentrators for Hubs: In multi-branch scenarios, configure IPsec concentrators at the hub to simplify policy management:

    config vpn ipsec concentrator     edit "branches"         set member "to_branch1" "to_branch2" "to_branch3"     next end

Frequently Asked Questions (FAQ)

Can I configure policy-based IPsec through the FortiGate GUI in FortiOS 7.x?

No. GUI configuration for policy-based IPsec is deprecated in FortiOS 7.x. While older documentation references unchecking "Enable IPsec Interface Mode" in the IPsec Wizard, current versions require exclusive CLI configuration. Enable the feature via Feature Visibility first, then configure Phase 1/Phase 2 and policies through CLI.

Why can't I see the "Policy based IPsec VPN" option in Feature Visibility?

This occurs when FortiGate operates in Policy-based NGFW mode. Policy-based IPsec VPN is only available in Profile-based NGFW mode. To use policy-based IPsec, switch NGFW mode via System → Settings → Operation Mode (requires reboot) or implement route-based VPN instead.

Do I need matching implementation types (policy vs. route-based) on both tunnel endpoints?

No. IPsec tunnel negotiation depends solely on matching Phase 1/Phase 2 parameters—not implementation type. You can safely deploy route-based IPsec locally while connecting to third-party policy-based implementations. The tunnel establishment process remains identical.

Why do I need multiple Phase 2 configurations for multiple subnets?

Policy-based IPsec requires explicit traffic selectors for each subnet pair when interoperating with third-party devices. FortiGate cannot automatically expand address groups into the multiple security associations required. For N local subnets and M remote subnets, you need N×M Phase 2 configurations—a significant scalability limitation avoided by route-based implementations.

Can policy-based IPsec work with SSL VPN?

No. SSL VPN interfaces are fundamentally incompatible with policy-based IPsec tunnels. Organizations requiring both technologies must implement route-based IPsec (phase1-interface) to enable integration between SSL and IPsec VPN services.

How do I troubleshoot a policy-based tunnel that won't establish?

Follow this diagnostic sequence:

  1. Verify Phase 1 parameters match on both ends (encryption, authentication, DH group)
  2. Confirm firewall policy destination interface matches Phase 1 source interface
  3. Check pre-shared key accuracy (case-sensitive)
  4. Validate remote gateway IP address reachability
  5. Use debugging commands: 
    diagnose vpn tunnel list          # Verify tunnel state diagnose debug flow filter addr <IP> diagnose debug flow trace start 100 diagnose debug enable

What's the maximum number of Phase 2 selectors per tunnel?

FortiGate recommends limiting Phase 2 selectors to fewer than 255 address objects per selector. Beyond this threshold, create additional Phase 2 configurations within the same tunnel rather than overloading a single selector.

Should I use policy-based or route-based for new deployments?

Route-based is strongly recommended for all new deployments unless you have specific requirements for L2TP-over-IPsec or must support identical subnets across sites. Route-based offers superior scalability, simplified troubleshooting, SSL VPN integration, and compatibility with modern networking practices including dynamic routing protocols over VPN tunnels.

Conclusion: The Evolving Role of Policy-Based IPsec

While policy-based IPsec VPN remains technically supported on FortiGate platforms, its role has diminished significantly in modern network architectures. The combinatorial complexity of Phase 2 configurations, incompatibility with SSL VPN, and NGFW mode restrictions make it a legacy solution suitable only for specific edge cases.

Fortinet's documentation trajectory confirms this shift—GUI configuration paths deprecated, CLI becoming the sole configuration method, and official guidance consistently recommending route-based implementations for flexibility and operational simplicity. Organizations maintaining policy-based configurations should evaluate migration paths to route-based architectures during their next security infrastructure refresh cycle, unless constrained by immutable third-party requirements.

For administrators inheriting policy-based implementations, understanding the interface-matching constraints, bidirectional initiation requirements, and troubleshooting methodologies remains essential—while simultaneously planning strategic migration to route-based architectures that align with contemporary zero-trust and SASE networking principles.