Given a short exact sequence of groups with certain conditions,
1
→
F
→
G
→
H
→
1
1\rightarrow F\rightarrow G\rightarrow H\rightarrow 1
, we prove that
G
G
has solvable conjugacy problem if and only if the corresponding action subgroup
A
⩽
A
u
t
(
F
)
A\leqslant Aut(F)
is orbit decidable. From this, we deduce that the conjugacy problem is solvable, among others, for all groups of the form
Z
2
⋊
F
m
\mathbb {Z}^2\rtimes F_m
,
F
2
⋊
F
m
F_2\rtimes F_m
,
F
n
⋊
Z
F_n \rtimes \mathbb {Z}
, and
Z
n
⋊
A
F
m
\mathbb {Z}^n \rtimes _A F_m
with virtually solvable action group
A
⩽
G
L
n
(
Z
)
A\leqslant GL_n(\mathbb {Z})
. Also, we give an easy way of constructing groups of the form
Z
4
⋊
F
n
\mathbb {Z}^4\rtimes F_n
and
F
3
⋊
F
n
F_3\rtimes F_n
with unsolvable conjugacy problem. On the way, we solve the twisted conjugacy problem for virtually surface and virtually polycyclic groups, and we give an example of a group with solvable conjugacy problem but unsolvable twisted conjugacy problem. As an application, an alternative solution to the conjugacy problem in
A
u
t
(
F
2
)
Aut(F_2)
is given.