Sometimes when we load a compiled eBPF program, the eBPF verifier will indicate that there is a type error in the program causing the program to fail to load. This article documents a solution to this error.
Example of error¶
For example, when this eBPF program
SEC("iter/bpf_sk_storage_map")
int iter__bpf_sk_storage_map(struct bpf_iter__bpf_sk_storage_map *ctx)
{
if (ctx->sk)
bpf_sk_storage_delete(&sk_storage_map, ctx->sk);
return 0;
}
is loaded into the kernel with the following type of error:
libbpf: prog 'iter__bpf_sk_storage_map': BPF program load failed: Permission denied libbpf: prog 'iter__bpf_sk_storage_map': -- BEGIN PROG LOAD LOG -- R1 type=ctx expected=fp ; if (ctx->sk) 0: (79) r2 = *(u64 *)(r1 +16) ; if (ctx->sk) 1: (15) if r2 == 0x0 goto pc+4 R1=ctx(id=0,off=0,imm=0) R2_w=ptr_sock(id=0,off=0,imm=0) R10=fp0 ; bpf_sk_storage_delete(&sk_storage_map, ctx->sk); 2: (79) r2 = *(u64 *)(r1 +16) ; bpf_sk_storage_delete(&sk_storage_map, ctx->sk); 3: (18) r1 = 0xffffa0658305aa00 5: (85) call bpf_sk_storage_delete#108 R2 type=ptr_or_null_ expected=ptr_ processed 5 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 -- END PROG LOAD LOG -- libbpf: prog 'iter__bpf_sk_storage_map': failed to load: -13 libbpf: failed to load object 'main.bpf.o' failed to load BPF object: permission denied
Solution¶
This error message has two key errors, one error is
R1 type=ctx expected=fp ; if (ctx->sk) 0: (79) r2 = *(u64 *)(r1 +16) ; if (ctx->sk) 1: (15) if r2 == 0x0 goto pc+4
Where R1 type=ctx expected=fp means that the verifier expects R1 to be of type fp and not ctx. By fp, we mean the type of the pointer on the stack, i.e., R1 is expected to be data on the stack and not ctx.
Another error is
R1=ctx(id=0,off=0,imm=0) R2_w=ptr_sock(id=0,off=0,imm=0) R10=fp0 ; bpf_sk_storage_delete(&sk_storage_map, ctx->sk); 2: (79) r2 = *(u64 *)(r1 +16) ; bpf_sk_storage_delete(&sk_storage_map, ctx->sk); 3: (18) r1 = 0xffffa0658305aa00 5: (85) call bpf_sk_storage_delete#108 R2 type=ptr_or_null_ expected=ptr_
Where R2 type=ptr_or_null_ expected=ptr_ means that the verifier expects R2 to be of type ptr and not prt_or_null, that is, it expects R2 to be a pointer and not a pointer or NULL. Here you may be a little confused, the previous judgment if (ctx->sk) has already ensured that it will not be NULL, why here still think it may be NULL, this is because the previous if judgment is not a stack variable, there is R1 type=ctx expected=fp problem also can not guarantee that it is not NULL.
The solution is simple: use a temporary variable to store the value of ctx->sk, and then use this temporary variable on the stack for subsequent operations:
SEC("iter/bpf_sk_storage_map")
int iter__bpf_sk_storage_map(struct bpf_iter__bpf_sk_storage_map *ctx)
{
- if (ctx->sk)
- bpf_sk_storage_delete(&sk_storage_map, ctx->sk);
+ struct sock *sk = ctx->sk;
+ if (sk)
+ bpf_sk_storage_delete(&sk_storage_map, sk);
return 0;
}
Meaning of common types of keywords¶
The specific meaning of common type keywords like the previous fp is documented here.
| keyword | describe |
|---|---|
| scalar | reg doesn't contain a valid pointer |
| ctx | reg points to bpf_context |
| map_ptr | reg points to struct bpf_map |
| map_value | reg points to map element value |
| map_value_or_null | points to map elem value or NULL |
| map_key | reg points to a map element key |
| fp | reg == frame_pointer + offset |
| pkt | skb->data |
| pkt_meta | skb->data - meta_len |
| pkt_end | skb->data + headlen |
| sock | reg points to struct bpf_sock |
| sock_or_null | reg points to struct bpf_sock or NULL |
| sock_common | reg points to sock_common |
| sock_common_or_null | reg points to sock_common or NULL |
| tcp_sock | reg points to struct tcp_sock |
| tcp_sock_or_null | reg points to struct tcp_sock or NULL |
| tp_buffer | reg points to a writable raw tp's buffer |
| xdp_sock | reg points to struct xdp_sock |
| ptr_ | points to a kernel struct that does not need to be null checked by the BPF program. |
| ptr_or_null_ | points to a kernel struct that has not been checked for null |
| dynptr_ptr | pointer to bpf_dynptr |
| mem | reg points to valid memory region |
| mem_or_null | reg points to valid memory region or NULL |
| buf | reg points to a read/write buffer |
| func | reg points to a bpf program function |
| inv | reg doesn't contain a valid pointer |
| flow_keys | reg points to bpf_flow_keys |
| percpu_ptr_ | reg points to a percpu kernel variable |
| rdonly_buf | reg points to a readonly buffer |
| rdonly_buf_or_null | reg points to a readonly buffer or NULL |
| rdwr_buf | reg points to a read/write buffer |
| rdwr_buf_or_null | reg points to a read/write buffer or NULL |
Comments